Functional protein classification for pandemic research

Abstract

A protein searcher includes a pre-trained CNN, a feature extractor, a database, and a KNN searcher. The pre-trained CNN trained on a previously classified amino acid database receives an unidentified amino acid sequence. The feature extractor extracts a feature vector of the unidentified amino acid sequence as a query feature vector. The database stores feature vectors of trained amino acid sequences and at least one untrained amino acid sequence, and stores associated classes of the trained amino acid sequences and associated labels of the at least one untrained amino acid sequence. The KNN searcher finds K feature vectors in the database that are close to the query feature vector, and outputs the associated class or the associated label of each of the K feature vectors.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application 63 / 086,086, filed October 1, 2020, which is incorporated herein by reference. Technical Field

[0003] In summary, this invention relates to protein classification. Background Technology

[0004] Proteins are made up of a series of amino acids, which are the building blocks of proteins. There are approximately 20 different possible amino acids in total.

[0005] In genetics, predicting protein function based on the amino acid sequence that makes up proteins is crucial. To advance biotechnology, it is important to identify proteins that catalyze novel reactions, bind specific microbial targets, or co-create new molecules.

[0006] Examples of protein functions include enzymes, ligand-binding proteins, membrane receptors, and membrane transporters. Labeling protein functions is crucial for understanding the cellular mechanisms in which they operate, identifying diseases caused by changes in gene or protein function, and discovering new tools for disease prevention, diagnosis, and treatment. In pandemic research, it is important to study the genomes of viruses that cause pandemics (such as Covid-19) to better understand the mechanisms of the disease and improve treatments for viral infections.

[0007] Viral DNA sequencing data is accumulating rapidly, necessitating the use of improved or alternative gene search tools to annotate, process, and mine this vast amount of data. Current methods are based on the BLASTp algorithm, but a significant portion of microbial proteins cannot be annotated through alignment, which is the core principle of these methods. Furthermore, alignment-based methods are known to have long runtimes. This approach is not scalable, as it grows with the size of the database, which itself grows exponentially in a short period. These models are also limited by long pipelines.

[0008] The paper "Using deep learning to annotate the protein universe" by Maxwell L. Bileschi et al. (bioRxiv, 2019) presents a novel approach to improving both the accuracy and runtime of annotation algorithms, and is based on deep learning methods that have shown groundbreaking performance in multiple fields such as computer vision and natural language processing. This paper describes the "ProtCNN" deep learning system, which manipulates proteins and has multiple "residual blocks," each based on a ResNet convolutional neural network (CNN) architecture.

[0009] The input to the neural network is an amino acid sequence, represented by 20 different symbols. For example, an exemplary protein can be defined by the following sequence:

[0010] MEVFK AP PIG

[0011] In the system described in the article, each symbol in such a sequence is converted into a 20-dimensional one-hot vector based on its value. For example, if M is defined as the 10th type of amino acid, then the 10th element of its associated vector will have a 1, and all other elements will have a 0. The associated sequence is a set of amino acid vectors arranged in the order of the protein sequence.

[0012] Before the sequences are fed into the ProtCNN network, the network is trained on a large protein database, which is divided into training, testing, and development sets (i.e., sets used to tune the network's hyperparameters).

[0013] As is standard for neural networks, a training set (i.e., a set of amino acid sequences labeled with the functions of proteins pre-classified) is used to train the network (also known as "labeling"). At the end of the training process, a test set is provided to the trained network. If the network is properly trained, it should output the known protein functions of the test set. Otherwise, the test set is used to further train the network. After this, a development set is used to further expand the operating parameters of the trained neural network.

[0014] The article also describes various improvements. One of them is a combined ProtCNN-kNN approach, where the output of ProtCNN is provided to the K nearest neighbor classifier in a 'single-sample' manner. The single-sample process is as follows:

[0015] 1) Features extracted from the ProtCNN network (i.e., the network's output before its last layer) are called "embedded features".

[0016] 2) Calculate the average embedding of each protein family used in training.

[0017] 3) Perform a cosine similarity search on the embedding features of untrained proteins in the dataset and known proteins.

[0018] The result is an untrained protein family, thus providing its annotation. As mentioned in the article, this method is particularly useful for small families with few feature vectors in the training set. Summary of the Invention

[0019] Therefore, according to a preferred embodiment of the present invention, a protein search engine is provided, comprising a pre-trained convolutional neural network (CNN), a feature extractor, a database, and a K-nearest neighbor (KNN) searcher. The pre-trained CNN is trained on a previously classified amino acid database and receives unidentified amino acid sequences. The feature extractor extracts feature vectors from the unidentified amino acid sequences as query feature vectors. The database stores feature vectors of trained amino acid sequences and feature vectors of at least one untrained amino acid sequence, and stores associated classes of the trained amino acid sequences and associated labels of at least one untrained amino acid sequence. The KNN searcher finds K feature vectors in the database that are close to the query feature vector and outputs the associated class or associated label for each of the K feature vectors.

[0020] Furthermore, according to a preferred embodiment of the invention, the pre-trained CNN is partially trained.

[0021] Furthermore, according to a preferred embodiment of the invention, at least one of the untrained amino acid sequences is from the SARS-CoV-2 virus, while the unidentified amino acid sequence is from a variant of the SARS-CoV-2 virus.

[0022] Furthermore, according to a preferred embodiment of the present invention, the operation being performed is a zero-sample search.

[0023] Furthermore, according to a preferred embodiment of the invention, the database and the KNN searcher are implemented in an association processing unit (APU) that includes a memory array for storing data and performing searches thereon.

[0024] Furthermore, according to a preferred embodiment of the invention, the memory array includes a vector portion, a class portion, and a KNN searcher portion. The vector portion stores feature vectors of trained amino acid sequences and feature vectors of at least one untrained amino acid sequence. The class portion stores associated classes and associated tags. The KNN searcher portion searches the vector portion to find K feature vectors and outputs the associated classes or associated tags of the K feature vectors.

[0025] Furthermore, according to a preferred embodiment of the invention, the memory array includes columns, and each column is used to store one of the feature vectors in the vector portion of the column, and to store the associated class or associated tag of one of the feature vectors in the class portion of the column.

[0026] According to a preferred embodiment of the present invention, a method for protein search is also provided. The method includes: training a pre-trained CNN on a previously classified amino acid database to receive unidentified amino acid sequences; extracting feature vectors of the unidentified amino acid sequences as query feature vectors; storing the feature vectors of the trained amino acid sequences and feature vectors of at least one untrained amino acid sequence; storing the associated classes of the trained amino acid sequences and the associated tags of the at least one untrained amino acid sequence; searching the database for K feature vectors that are close to the query feature vector; and outputting the associated class or associated tag for each of the K feature vectors.

[0027] Furthermore, according to a preferred embodiment of the present invention, the method includes: partially training a pre-trained CNN.

[0028] Furthermore, according to a preferred embodiment of the invention, at least one of the untrained amino acid sequences is from the SARS-CoV-2 virus, while the unidentified amino acid sequence is from a variant of the SARS-CoV-2 virus.

[0029] Furthermore, according to a preferred embodiment of the present invention, the operation being performed is a zero-sample search.

[0030] Additionally, according to a preferred embodiment of the present invention, the method includes: performing the first and second steps of storage, and the step of searching in an association processing unit (APU), the APU including a memory array having columns divided into vector portions and class portions.

[0031] Finally, according to a preferred embodiment of the invention, the first step of storage includes: storing one of the feature vectors in the vector portion of a column of the column, and the second step of storage includes: storing the associated class or associated tag of one of the feature vectors in the class portion of a column of the column. Attached Figure Description

[0032] The subject matter considered to be of the present invention is specifically pointed out and explicitly claimed at the end of the specification. However, the invention, with respect to its organization and operation methods, together with its objects, features and advantages, can be best understood by referring to the following detailed description when reading the accompanying drawings, wherein:

[0033] Figure 1A This is a schematic diagram of the first operational stage of a protein search system constructed and operated according to a preferred embodiment of the present invention;

[0034] Figure 1B yes Figure 1A A schematic diagram of the second operational phase of the protein search system; and

[0035] Figure 1C yes Figure 1A A schematic diagram of the third operational stage of the protein search system.

[0036] It should be understood that, for the sake of simplicity and clarity, the elements shown in the figures need not be drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Furthermore, where deemed appropriate, reference numerals may be repeated in the figures to indicate corresponding or similar elements. Detailed Implementation

[0037] In the detailed description below, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention can be practiced without these specific details. In other instances, well-known methods, processes, and components have not been described in detail to avoid obscuring the invention.

[0038] The applicant has recognized that speed is crucial in the face of pandemics such as COVID-19. The earlier proteins are functionally labeled, the sooner we can understand the genomes and protein structures of viruses and bacteria.

[0039] The applicant also recognized that deep learning methods are significantly accelerated when implemented in an Association Processing Unit (APU). The APU can also handle larger databases, such as protein databases. Furthermore, the applicant has realized that deep learning methods can be further improved by using "zero-shot" learning to learn proteins that are not visible during training.

[0040] Now for reference Figure 1A , Figure 1B and Figure 1C This illustrates three operational phases of a protein search system 10 constructed and operated according to a preferred embodiment of the invention. The protein search system 10 includes a pre-trained convolutional neural network (CNN) 12, a feature extractor 14, a tagger 16, and an APU 18.

[0041] APU 18 can be any suitable APU, such as the Gemini APU available from GSI Technology Inc. in California, USA, and can be formed from a memory array capable of both storing data and performing searches on it. Figure 1A As shown, APU 18 can store data in its data column 24.

[0042] The pre-trained CNN 12 can be any suitable CNN, such as the ProtCNN neural network, and can be trained on an appropriate classification protein database 11 (e.g., the PFAM database). The pre-trained CNN 12 can receive amino acid sequences as input from the database 11 and can be trained to provide protein classes, functions, or labels also stored in the database 11 as outputs.

[0043] Feature extractor 14 can be any suitable feature extractor designed to extract the feature vector or "embedding" of the trained protein from the output of the layers preceding the last layer of the pre-trained CNN 12 and the output of the pre-trained CNN 12 (i.e. the protein class, function, or label associated with the extracted feature vector).

[0044] In the initial phase of the protein search system 10's operation, the database 11 can provide multiple amino acid sequences to the pre-trained CNN 12 and their associated classes to the APU 18. For each amino acid sequence, the feature extractor 14 can store the extracted feature vector in a separate column 24i of the APU 18. Figure 1A As shown, APU 18 can be divided into a vector portion 20 for storing feature vectors and a class portion 22 for storing classes. Each column 24i of APU 18 can store information (feature vector and class) associated with an input amino acid sequence. For each amino acid sequence, database 11 can provide an associated class to be stored in its associated column 24i.

[0045] Typically, a large portion of the database 11 can be provided to the protein search system 10 so that its feature vectors and classes can be stored in columns 24i of the APU 18. For example, the APU 18 can store one million feature vectors and classes generated from the PFAM database. The APU 18 can also store even larger databases.

[0046] In the second phase of the operation and as Figure 1B As shown, the protein search system 10 can receive new amino acid sequences, such as the amino acid sequence of a new virus, and can also receive the name of the new sequence. For example, the name of the new virus might be "COVID" or "COVID version Delta".

[0047] The protein search system 10 can provide new amino acid sequences to the pre-trained CNN 12, which can operate on them and generate classes. However, since the amino acid sequences are new, the generated classes are unlikely to be correct. Regardless of the generated classes, the feature extractor 14 can extract feature vectors from the new amino acid sequences and add them to a new column, denoted here as 24j, in the vector portion 20 of the APU 18.

[0048] The protein search system 10 can also provide the received names to the tagger 16, which can generate tags for new viruses based on the received names. The feature extractor 14 can provide column number 24j to the tagger 16, in which the feature vector of the new amino acid sequence is stored, and thus the tagger 16 can store the new virus tags in the same column, i.e., column 24j, but in class section 22.

[0049] It should be understood that, Figure 1B In the second stage shown, the protein search system 10 can avoid retraining the pre-trained CNN 14 when a new amino acid sequence is received. Instead, it can use the previous training to generate embeddings for the new amino acid sequence. However, since the pre-trained CNN 12 has not been retrained, the protein search system 10 may only have labels; because the classes generated by the pre-trained CNN 12 will be incorrect.

[0050] In the third phase of the operation and as Figure 1C As shown, the protein search system 10 can receive additional novel amino acid sequences that have not yet been identified, such as novel amino acid sequences from newly discovered viruses or novel amino acid sequences from newly discovered animal proteins. The latter may help determine how proteins evolve.

[0051] The protein search system 10 can provide unrecognized amino acid sequences to the pre-trained CNN 12. The CNN 12 can operate on the unrecognized amino acid sequences and generate classes for them. However, since the amino acid sequence is new, just like the amino acid sequence from the previous stage, the generated class is unlikely to be correct. Regardless of the generated class, the feature extractor 14 can extract feature vectors for the unrecognized amino acid sequences. However, since other new amino acid sequences come from newly discovered sources, such as viruses or animals, no labels may be generated for them.

[0052] Conversely, the protein search system 10 can perform a 'zero-sample search' and can search the data stored in column 24 of APU 18 to find matches or nearest neighbors to the feature vectors of unidentified amino acid sequences. At this stage, the feature vectors of unidentified amino acid sequences can serve as query vectors for the similarity searcher 30, which is implemented in the similarity search section of APU 18 and operates on the data stored in column 24 of the feature section 20.

[0053] Therefore, as Figure 1A and Figure 1B As shown, the first and second stages can generate classified proteins. Figure 1A A database of feature vectors of ) and labeled but unclassified proteins ( Figure 1B A database of feature vectors. This database can be searched in the third stage. Figure 1C The search can be a zero-shot search, which is a search using feature vector data from a pre-trained CNN, which includes both classified and unclassified data, even though it has been labeled.

[0054] For zero-shot search, the similarity searcher 30 can be a K-nearest neighbor (KNN) searcher, which can perform the similarity search using cosine, L1, or Hammond similarity metrics. An exemplary similarity search is described in U.S. Patent 10,929,751, filed February 23, 2021, entitled "Finding K Extreme Values ​​in Constant Processing Time," and U.S. Patent Application 16 / 033,259, filed July 12, 2018, entitled "Natural Language Processing With KNN," both of which are commonly owned by the applicants of this invention and are incorporated herein by reference.

[0055] The similarity searcher 30 can find the K most similar vectors to the query vector from the entire database, where K can be 5, 10, or 50, or any other size significantly smaller than the database size, and can output association data (class and / or label) stored in the class portion 22 of the searched vectors. Therefore, the protein search system 10 can determine whether a newly discovered virus belongs to the same family as an untrained but known virus (whose feature vector is generated in the second stage) and / or whether it is similar to other amino acid sequences in the database.

[0056] It should be understood that because the similarity searcher 30 can be implemented in the APU 18, it can be an in-memory processor and can perform its search in parallel on all columns 24. As described in US 10,929,751, due to parallel operation, the similarity searcher 30 may have a complexity of O(1), so adding additional data to the other columns 24 may not affect its operating speed.

[0057] It should be understood that the second phase can be executed multiple times, resulting in multiple labeled feature vectors in addition to the large number of class-specific feature vectors from the first phase. Furthermore, these additional feature vectors may not affect the operating speed of the similarity searcher 30.

[0058] The applicant has recognized that the pre-trained CNN 12 can be fully trained, or it can be only partially trained, as described in U.S. Patent 10,929,751. In this alternative embodiment, the pre-trained CNN 12 may have stopped training when it began to converge (i.e., when its results began to be mostly correct). The remaining components of the protein search system 10 can remain unchanged. Thus, the feature extractor 14 can extract “partially trained” feature vectors from the pre-trained CNN 12 in its partially trained state for classified amino acid sequences, labeled amino acid sequences, and for the query amino acid sequence, and the similarity searcher 30 can perform its KNN search operation between the partially trained feature vectors of the query protein and the partially trained feature vectors stored in the APU 18.

[0059] It should be understood that, since the similarity searcher 30 can be implemented on the APU 18, it can find the K closest classes or functions of the query amino acid sequence in less than 1 millisecond per sequence. Furthermore, the protein search system 10 avoids retraining the pre-trained CNN 12 when identifying new amino acid sequences, which would require a long time and significant computing resources. Therefore, the protein search system 10 may be able to identify newly identifiable viral variants very quickly.

[0060] For example, the pre-trained CNN 12 can be trained based on the previously known amino acid sequence of the SARS-CoV-2 virus discovered in December 2019, and the associated feature vectors and their classes can be stored in APU 18 (first stage). In the second stage, the amino acid sequence of SARS-CoV-2 can be fed into the pre-trained CNN 12, and the feature extractor 14 can store the feature vector of this sequence in APU 18, labeled "COVID-19".

[0061] Although not mentioned above, the feature vector of the sequence can be provided as a query vector to the similarity searcher 30, and the similarity searcher 30 can perform a search to match the new sequence with existing sequences in order to find the functional information of proteins in the SARS-COV-2 sequence.

[0062] Furthermore, when a new virus is discovered (Phase 3), it can be fed into the protein search engine 10 to search an expanded database to determine whether the new virus is a variant of SARS-CoV-2. Since the similarity search engine 30 is implemented on the APU 18, the entire process can be completed in seconds.

[0063] It should be understood that, in alternative embodiments, the protein searcher 10 may utilize a database and a fast central processing unit (CPU) instead of the APU 18. In this embodiment, the CPU may perform a zero-shot KNN search on feature vector data from a pre-trained CNN 12, which includes classified data and unclassified but labeled data.

[0064] Unless otherwise expressly stated, it is evident from the foregoing discussion that, throughout this specification, the use of terms such as “processing,” “operation,” “calculation,” and “determining” refers to the actions and / or processes of any type of general-purpose computer (e.g., client / server systems, mobile computing devices, smart appliances, cloud computing units, or similar electronic computing devices) that manipulate and / or convert data within the registers and / or memory of a computing system into other data within the memory, registers, or other such information storage, transmission, or display devices of the computing system.

[0065] Embodiments of the present invention may include means for performing the operations described herein. Such means may be specifically constructed for a desired purpose, or may include a computing device or system generally having at least one processor and at least one memory that can be selectively activated or reconfigured by a computer program stored in a computer. When instructed by software, the resulting means can transform a general-purpose computer into an inventive element as discussed herein. Instructions may define the operation of the inventive device with a desired computer platform. Such a computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including optical disks, magneto-optical disks, read-only memory (ROM), volatile and non-volatile memory, random access memory (RAM), electrically programmable read-only memory (EPROM), electrically erasable and programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, disks on keys, or any other type of medium suitable for storing electronic instructions and capable of being coupled to a computer system bus. Computer-readable storage media may also be implemented in cloud storage.

[0066] Some general-purpose computers may include at least one communication element to enable communication with data networks and / or mobile communication networks.

[0067] The processes and displays presented herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used in accordance with the teachings and procedures herein, or they may prove convenient for constructing more specialized devices for performing the desired methods. Various desired structures of these systems will emerge from the following description. Furthermore, embodiments of the invention are described without reference to any particular programming language. It will be understood that various programming languages ​​can be used to implement the teachings of the invention described herein.

[0068] Although certain features of the invention have been described and illustrated herein, many modifications, substitutions, alterations, and equivalents will now occur to those skilled in the art. Therefore, it should be understood that the appended claims are intended to cover all such modifications and variations that fall within the true spirit of the invention.

Claims

1. A protein search engine, comprising: The pre-trained convolutional neural network (CNN) is trained on a previously classified amino acid database to receive unrecognized amino acid sequences; A feature extractor is used to extract the feature vector of the unidentified amino acid sequence as a query feature vector; A database for storing feature vectors of trained amino acid sequences and feature vectors of at least one untrained amino acid sequence, and for storing associated classes of the trained amino acid sequences and associated tags of the at least one untrained amino acid sequence. as well as A K-Nearest Neighbor (KNN) searcher is used to find K feature vectors in the database that are close to the query feature vector, and to output the associated class or the associated label of each of the K feature vectors.

2. The protein searcher according to claim 1, wherein, The pre-trained convolutional neural network is partially trained.

3. The protein searcher according to claim 1, wherein, One of the at least one untrained amino acid sequences is derived from the SARS-CoV-2 virus, and the unidentified amino acid sequence is derived from a variant of the SARS-CoV-2 virus.

4. The protein searcher according to claim 1, wherein, The operation performed is a zero-shot search.

5. The protein searcher according to claim 1, wherein, The database and the K-nearest neighbor searcher are implemented in an association processing unit, which includes a memory array for storing data and performing searches on the data.

6. The protein searcher according to claim 5, wherein, The memory array includes: The vector portion is used to store the feature vectors of the trained amino acid sequence and the feature vectors of the at least one untrained amino acid sequence. The class portion, which stores the associated class and the associated tag; and The K-nearest neighbor searcher part is used to search the vector part to find the K feature vectors and to output the associated class or the associated label of the K feature vectors.

7. The protein searcher according to claim 6, wherein, The memory array includes columns, and each column is used to store a feature vector of the feature vector in the vector portion of the column, and to store the associated class or the associated tag of the feature vector of the feature vector in the class portion of the column.

8. A method for protein search, the method comprising: The pre-trained convolutional neural network (CNN) is trained on a previously classified amino acid database to receive unrecognized amino acid sequences; Extract the feature vector of the unidentified amino acid sequence as the query feature vector; Store the feature vectors of trained amino acid sequences and the feature vectors of at least one untrained amino acid sequence; The system stores the associated classes of the trained amino acid sequences and the associated tags of the at least one untrained amino acid sequence. Find the K feature vectors in the database that are closest to the queried feature vector; as well as Output the associated class or associated label for each of the K feature vectors.

9. The method of claim 8, further comprising: The pre-trained convolutional neural network is partially trained.

10. The method according to claim 8, wherein, One of the at least one untrained amino acid sequences is derived from the SARS-CoV-2 virus, and the unidentified amino acid sequence is derived from a variant of the SARS-CoV-2 virus.

11. The method according to claim 8, wherein, The operation performed is a zero-shot search.

12. The method of claim 8, further comprising: The first and second steps of storage are implemented, as well as the step of searching in the association processing unit, which includes a memory array having columns divided into vector portions and class portions.

13. The method of claim 12, wherein, The first step of the storage includes storing one of the feature vectors in the vector portion of one column of the column, and the second step of the storage includes storing the associated class or the associated tag of the one feature vector in the feature vector in the class portion of the column of the column.

Images