Protein language construction method and device, electronic equipment and computer program product

Abstract

The invention is suitable for the technical field of computers, and provides a protein language construction method and device, electronic equipment and a computer program product. The method comprises the following steps: segmenting a target protein based on secondary structure information to obtain more than two target protein fragments; determining a protein language word corresponding to each target protein fragment; and determining a protein language statement corresponding to the target protein based on the protein language word corresponding to each target protein fragment. According to the method and the device, the protein language model can capture semantic information of a protein molecule level more easily, so that better protein representation is obtained.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a protein language construction method, device, electronic device and computer program product. Background Art

[0002] Protein is a biological macromolecule composed of 20 standard amino acids. The complexity of an organism is reflected by the number of genes and proteins related to life activities. Static gene sequences are difficult to fully reveal the essential laws of life and the occurrence and development of diseases. As a gene product, protein has both dynamic and spatiotemporal characteristics and participates in many cellular activities including material transport, energy metabolism, and immune catalysis. Research related to protein function can not only promote the development of life sciences, but also help researchers more accurately determine the causes of diseases, find corresponding therapeutic targets, and promote the development of new drugs.

[0003] Traditional protein research relies on experimental methods, such as using cryo-electron microscopy to analyze the three-dimensional structure of proteins and using biological wet experiments to detect protein functions. However, these methods are often time-consuming and costly, and cannot perform large-scale data detection. Relatively speaking, computational methods are efficient and low-cost, and have become an important supplement to biological wet experiments.

[0004] At present, most machine learning models and mathematical statistics methods cannot process raw protein sequences and structures. Therefore, for protein-related research based on computational methods, protein representation is a basic and critical step, which directly affects the performance of protein-related downstream tasks. At present, many protein representation learning methods regard protein sequences as a language, use natural language processing as a technical route, and use protein language models to learn protein representation. In recent years, researchers have begun to use protein language models to learn the intrinsic laws of proteins, and have achieved better representation results than traditional methods. To use language models to learn protein representation, proteins must first be converted into a language. At present, most studies define protein language by treating amino acids (AA) as words and amino acid sequences as sentences.

[0005] Although the representation learning model based on this fine-grained protein language has achieved better representation results than traditional experience-based methods, the information carried by using only amino acid sequences as protein language is very limited. In life activities, the structure of a protein often determines its function, and structural information plays an important role in learning the intrinsic characteristics of a protein. Studies have shown that representation methods based on protein structure are superior to sequence-based methods with less data.

[0006] In this regard, some studies have attempted to design a protein language that contains structural information. This method combines local residue conformations with amino acids as words to construct a protein language that integrates sequence and structure. However, this language is similar to the fine-grained protein sequence language with amino acids as words mentioned above, and is still limited to the amino acid level, which is not conducive to the protein language model capturing protein semantic information and cannot obtain better protein representation. Summary of the invention

[0007] The embodiments of the present application provide a protein language construction method, device, electronic device and computer program product, which are more conducive to the protein language model to capture the semantic information at the protein molecular level, thereby obtaining a better protein representation.

[0008] In a first aspect, the present application provides a method for constructing a protein language, comprising:

[0009] The target protein is segmented based on the secondary structure information to obtain two or more target protein fragments;

[0010] Determine the protein language words corresponding to each target protein fragment;

[0011] Based on the protein language words corresponding to the target protein fragments, a protein language sentence corresponding to the target protein is determined.

[0012] In an embodiment of the present application, by segmenting the target protein based on the secondary structure information, more than two target protein fragments can be obtained, and by determining the protein language words corresponding to each target protein fragment, the protein language sentence corresponding to the target protein can be determined based on the protein language words corresponding to each target protein fragment, thereby realizing the construction of the protein language. Compared with the fine-grained protein language using amino acids as words, this solution segments the target protein into more than two coarse-grained target protein fragments based on the secondary structure information, so that the protein language sentence obtained based on the coarse-grained target protein fragments is shorter, which is more conducive to the protein language model capturing the semantic information at the protein molecular level, and the coarse-grained protein language words can better characterize the local conserved domain of the target protein, thereby helping the protein language model learn the features at the molecular level and ultimately obtaining a better protein representation.

[0013] In a second aspect, the present application provides a protein language construction device, comprising:

[0014] A target segmentation module is used to segment the target protein based on the secondary structure information to obtain two or more target protein fragments;

[0015] A word determination module, used to determine the protein language words corresponding to each target protein fragment;

[0016] The sentence determination module is used to determine the protein language sentence corresponding to the target protein based on the protein language words corresponding to each target protein fragment.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method described in the first aspect above.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method described in the first aspect above is implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, wherein when the computer program is executed, the method described in the first aspect above is executed.

[0020] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 It is a schematic diagram of the process of constructing a protein language provided in the embodiments of the present application;

[0023] Figure 2 It is a flow chart of the method for determining protein language words provided in the embodiment of the present application;

[0024] Figure 3 It is a schematic diagram of the construction of the protein language vocabulary;

[0025] Figure 4 It is a schematic diagram of the conversion of proteins into protein language sentences;

[0026] Figure 5 It is a schematic diagram of the structure of the protein language construction device provided in the embodiment of the present application;

[0027] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0029] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0030] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0031] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0033] Since protein functions are often related to the local conserved domains present therein, the structures of these regions in similar proteins remain highly similar or hardly change. Local conserved domains may be active sites, binding sites, or regulatory regions of proteins, which are essential for maintaining the functions of proteins. However, fine-grained languages ​​with single amino acids as words have limited performance in expressing these local conserved domains, which makes it difficult for language models to capture this feature from a semantic level. In order to better characterize the semantics of protein molecules and promote downstream protein function analysis, the present application embodiment takes into account the characteristics of the local conserved domains of the structure of proteins, proposes a coarse-grained protein language definition strategy for protein function analysis, and proves the advantages of this coarse-grained language over the fine-grained language of amino acids, as shown in Table 1 below.

[0034] The protein language construction method provided in the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0035] See also Figure 1 , Figure 1 The flowchart of the protein language construction method provided in the embodiment of the present application is shown, which is applied to electronic devices. As an example but not limitation, the method includes the following steps:

[0036] Step 101 : segment the target protein based on the secondary structure information to obtain two or more target protein fragments.

[0037] Among them, the above secondary structure information is divided into 8 types, namely helix (H), bridge (B), extended chain (E), 3-helix (G), 5-helix (I), hydrogen bond turn (T), curl (S), loop or random curl (the official mark is blank, which can be customized in this application, for example, defined as C).

[0038] In one embodiment, the secondary structure information of the protein may be extracted using a Dictionary of Secondary Structure of Protein (DSSP) tool.

[0039] For the target protein and the protein samples below, the proteins are segmented based on the secondary structure information, including:

[0040] The amino acids belonging to the same secondary structure information in the protein sequence of the protein are divided into a protein fragment;

[0041] Wherein, when the protein is a target protein, the protein fragment is a target protein fragment; when the protein is a protein sample, the protein fragment is the protein sample fragment described below.

[0042] This embodiment divides the protein based on the secondary structure information, and obtains discrete protein fragments after division. Since most amino acids in proteins are arranged according to a certain spatial structure, a protein fragment will contain multiple amino acids. Therefore, the protein language obtained based on this segmentation method is coarse-grained. Coarse-grained words greatly reduce the length of protein sentences, thereby promoting the protein language model to capture contextual information and learn protein spatial semantics.

[0043] Through research and statistics, it is found that the number of fragments containing very few amino acids in the protein fragments obtained by segmentation is small, and the number of fragments with too long length is very small. And the fragments containing very few amino acids often do not have enough biological significance and may not provide enough information to describe the function or structure of the protein. The frequency of occurrence of fragments that are too long is not high, and the moderate fragment length ensures that each protein fragment has enough information, can provide a specific structure, function or pattern, and help the protein language model to effectively learn useful features without being disturbed by irrelevant noise or redundant information. Therefore, in order to prevent the noise fragments from affecting the learning effect of the subsequent protein language model as much as possible, in one embodiment, the length of each target protein fragment obtained after the target protein is segmented can be determined, and the target protein fragments with a length less than the first length threshold or a length greater than the second length threshold are filtered. Based on the target protein fragments with a length greater than or equal to the first length threshold and a length less than or equal to the second length threshold, step 102 is performed (i.e., determining the protein language words of the target protein fragments with a length greater than or equal to the first length threshold and a length less than or equal to the second length threshold). Among them, the length of the above-mentioned target protein fragment can refer to the number of amino acids in the target protein fragment. The above-mentioned first length threshold is less than the above-mentioned second length threshold. The first length threshold is used to screen small fragments containing very few amino acids. The second length threshold is used to screen protein fragments that are too long.

[0044] Step 102: determine the protein language words corresponding to each target protein fragment.

[0045] Since the target protein fragment is a coarse-grained protein fragment, the protein language words corresponding to the target protein fragment are coarse-grained language words. The coarse-grained language words based on secondary structure information in this embodiment can better describe the local conserved domain of protein, thereby helping the protein language model to learn the features at the molecular level and ultimately obtain better protein representation.

[0046] See also Figure 2 , the electronic device can determine the protein language words according to step 201 and step 202.

[0047] Step 201 : extracting the multi-source heterogeneous features of each target protein fragment.

[0048] The multi-source heterogeneous features are usually composed of features from multiple different sources. Optionally, the features from multiple different sources include but are not limited to amino acid composition features, evolutionary conservative features, and spatial geometric features.

[0049] In a possible implementation, for the target protein fragment and the protein sample fragment below, the multi-source heterogeneous features of the protein fragment are extracted, including:

[0050] Extract the amino acid composition characteristics of protein fragments;

[0051] Determine the evolutionary information spectrum of the protein corresponding to the protein fragment, and extract the evolutionary conservative features of the protein fragment from the evolutionary information spectrum based on the evolutionary information spectrum and the position of the protein fragment in the protein sequence of the protein; when the protein fragment is a target protein fragment, the protein is a target protein; when the protein fragment is a protein sample fragment, the protein is a protein sample;

[0052] Extract spatial geometric features of protein fragments;

[0053] The amino acid composition features, evolutionary conservation features and spatial geometric features are spliced ​​together to obtain the multi-source heterogeneous characteristics of protein fragments.

[0054] Since the percentage and number of standard amino acids in protein fragments can intuitively represent the content of standard amino acids in protein fragments, and thus characterize the amino acid composition of protein fragments, the amino acid composition characteristics of the above protein fragments can refer to the percentage of N standard amino acids in protein fragments, or the number of N standard amino acids in protein fragments. N is an integer greater than zero. Optionally, some or all of the standard amino acids can be selected from all the standard amino acids of the protein as the N standard amino acids according to the scenario requirements.

[0055] Taking the percentage of N standard amino acids in the protein fragment as the amino acid composition feature of the protein fragment can avoid normalization processing and improve data processing efficiency.

[0056] When the number of N standard amino acids in a protein fragment is used as the amino acid composition feature of the protein fragment, in order to improve data processing efficiency, the number of the N standard amino acids in the protein fragment may be normalized.

[0057] By way of example and not limitation, N is 20, and the 20 standard amino acids include glycine, alanine, valine, leucine, isoleucine, methionine (methionine), proline, tryptophan, serine, tyrosine, cysteine, phenylalanine, asparagine, glutamine, threonine, aspartic acid, glutamic acid, lysine, arginine, and histidine.

[0058] As an example but not a limitation, for a given protein fragment, assuming it contains n amino acids, the corresponding amino acid composition feature is represented by a vector v consisting of 20 elements a =[e1,e2,…,e 20 ], respectively indicating the percentage of 20 standard amino acids in the protein fragment i =f(P,A i ) / n,f(P,A i ) is the statistical value of the i-th amino acid A i The frequency of occurrence of the i-th amino acid A in the protein fragment P i The function of isotope ratio is the number of amino acids in a protein fragment P), where n is the length of the protein fragment (i.e., the number of amino acids in the protein fragment).

[0059] In one embodiment, two types of protein evolutionary information profiles can be used, including a Position-Specific Scoring Matrix (PSSM) profile and a Hidden Markov Models (HMM) profile. The PSSM profile can be generated by PSI-BLAST based on a protein search for a specified protein database. The HMM profile can be constructed by HHblits based on a protein search for a specified protein database.

[0060] It should be noted that, since the final forms of the PSSM spectrum and the HMM spectrum are matrices, the PSSM spectrum can be called a PSSM matrix, and the HMM spectrum can be called an HMM matrix.

[0061] The evolutionary conservative features of protein fragments can be extracted from the evolutionary information spectrum of the corresponding position. For example, if the position of a protein fragment in the protein sequence is [5, 10], that is, the 5th to 10th amino acids all belong to the protein fragment, then the information from the 5th to 10th rows should be intercepted from the evolutionary information spectrum of the protein. Before feature extraction, the PSSM spectrum and HMM spectrum need to be normalized to reduce the features with a larger numerical range to an appropriate range to control the scale of the data. For the HMM spectrum, the elements therein can be normalized according to the following operations:

[0062]

[0063] Among them, x1 represents the element in the HMM spectrum, Indicates the normalized value of the element, and * indicates infinity. If the element x1 in the HMM spectrum is infinite, the normalized value of the element is 1; if the element x1 in the HMM spectrum is not infinite, the normalized value of the element is

[0064] For the PSSM spectrum, the sigmoid function can be used to normalize the elements therein. The formula is as follows:

[0065]

[0066] Where x2 represents the element in the PSSM spectrum, Represents the normalized value of this element.

[0067] Both the evolutionary conservative features based on PSSM spectrum and the evolutionary conservative features based on HMM spectrum can be expressed as a matrix with a dimension of n*20, where n is the number of amino acids in the protein fragment. In order to facilitate subsequent feature splicing, the two evolutionary conservative features of the protein fragment are reduced in dimension, and the information of each column is summed and divided by the length of the protein fragment to obtain two 20-dimensional vectors. The evolutionary conservative features of the protein fragment are obtained by splicing these two 20-dimensional vectors.

[0068] This embodiment extracts spatial geometric information of protein fragments to give protein language richer structural semantics. Currently, spatial geometric information includes secondary structure information of structural fragments (i.e., protein fragments), which is obtained by the DSSP tool based on protein structure analysis. This embodiment uses an 8-dimensional one-hot vector to represent the 8 types of secondary structures used. More spatial geometric features can also be represented using information such as hemispherical exposure area and residue depth information. Optionally, by extracting residue-level structural information and using sequence models to learn global structural information of protein fragments, the structural features of protein fragments can be obtained as a supplement to spatial geometric features.

[0069] For each protein fragment, a 68-dimensional feature vector can be obtained by splicing the amino acid composition characteristics, evolutionary conservation characteristics and spatial geometric characteristics of the protein fragment.

[0070] Step 202 : Based on the multi-source heterogeneous characteristics of each target protein fragment, a protein language word corresponding to each target protein fragment is determined from a protein language vocabulary.

[0071] For each target protein fragment, the protein language words corresponding to the target protein fragment can be queried from the protein language vocabulary.

[0072] In order to avoid the problem of the protein language vocabulary space being too large, in one embodiment, a semantic space clustering strategy can be used to construct a protein language vocabulary of appropriate space size, and then the corresponding words of the protein structure fragments in the protein language vocabulary are determined through semantic space mapping, and finally the protein sentence is obtained through word order arrangement.

[0073] In a possible implementation, the protein language vocabulary is constructed by:

[0074] Segment each protein sample based on the secondary structure information to obtain two or more protein sample fragments corresponding to each protein sample;

[0075] Extracting multi-source heterogeneous features of each protein sample fragment;

[0076] Based on the multi-source heterogeneous characteristics of each protein sample fragment, each protein sample fragment is clustered to obtain K clusters, where K is an integer greater than 1;

[0077] Determine that the cluster indexes of the K clusters are protein language words corresponding to the representative structural fragments of the clusters, and the representative structural fragments of the clusters are the cluster centers of the clusters;

[0078] A protein language vocabulary is constructed based on the protein language words of the representative structural fragments of the K clusters.

[0079] The above protein samples refer to proteins used to construct a protein language vocabulary.

[0080] K is the number of protein language words in the protein language vocabulary, i.e., the number of words. The characteristics of protein fragments in a cluster are similar, so the protein language words corresponding to all protein fragments in a cluster are the same, i.e., a cluster can represent a protein language word, which can be identified using a cluster index. Exemplarily, K is 800, and there are 800 clusters, and the cluster indexes of these 800 clusters are 000-799. Assuming that a protein fragment is divided into a cluster with a cluster index of 008 after clustering, then 008 can be used to represent the protein fragment, and 008 is the protein language word of this protein fragment.

[0081] In this embodiment, an unsupervised clustering method can be used to summarize protein sample fragments, and all the protein sample fragment representations obtained above are clustered. Each cluster obtained by clustering (i.e., cluster cluster) is regarded as a word, so as to construct a protein structure language vocabulary with a specified vocabulary space size. The protein sample fragments belonging to a certain cluster are represented by corresponding words (i.e., cluster cluster index), and the number of cluster clusters is the size of the constructed protein language vocabulary.

[0082] In this embodiment, protein sample fragments can be clustered through two schemes: 1) using the classic K-means algorithm for clustering; 2) using Vector Quantized Variational Autoencoder (VQ-VAE) for clustering.

[0083] K-means is a classic unsupervised clustering method that can cluster protein sample fragments into K clusters. For example, in this embodiment, the value of K is set to 800, that is, the size K of the protein language vocabulary is 800.

[0084] VQ-VAE is a generative model that combines variational autoencoders and vector quantization techniques to learn discrete latent representations. This model is mainly used in the field of generation. In the encoding stage, through the vector quantization step, the latent representation of the sample can be mapped to the vector of the nearest neighbor in the codebook, achieving an implicit clustering effect, and can therefore also be used as a clustering method. During the training process, VQ-VAE will adaptively adjust the encoder and Codebook so that the vectors therein can fit the latent space representation, thereby realizing the discrete clustering of protein sample fragments. The size of the Codebook can be customized. For example, in this embodiment, the size of the Codebook is set to 1000, that is, the size K of the protein language vocabulary is 1000.

[0085] After the protein language vocabulary is constructed, the electronic device can convert the target protein into a coarse-grained language statement. Figure 3 Shown is a schematic diagram of the construction of the protein language vocabulary. Figure 3 The protein structure data set (ProteinDataBank, PDB) in the protein structure data set is used to provide protein samples; after the electronic device obtains each protein sample from the protein structure data set, it divides each protein sample using the protein structure segmentation method to obtain protein sample fragments of each protein sample; and performs feature extraction (i.e., Figure 3 The fragment feature extraction in ) obtains the multi-source heterogeneous features of each protein sample fragment (i.e. Figure 3 Based on the multi-source heterogeneous features and clustering method, each protein sample fragment is clustered to obtain K clusters (i.e. Figure 3 The clustering space in ) and the protein language words of representative structural fragments, which constitute the protein language vocabulary (i.e. Figure 3 The protein structure data set can also be referred to as a protein structure database. The protein structure segmentation method is a method for segmenting proteins based on secondary structure information provided in this embodiment.

[0086] In one possible implementation, when determining the protein language words corresponding to each target protein fragment from a protein language vocabulary, the electronic device can determine the cluster to which each target protein fragment belongs based on the multi-source heterogeneous characteristics of each target protein fragment; determine the protein language words of the representative structural fragments of the cluster to which each target protein fragment belongs from the protein language vocabulary, and determine the protein language words as the protein language words corresponding to the corresponding target protein fragment.

[0087] Since the protein language vocabulary is obtained by clustering the multi-source heterogeneous features of protein sample fragments, on this basis, for each target protein fragment, the target protein fragment can be clustered in the existing clustering space (i.e., cluster cluster) based on the multi-source heterogeneous features of the target protein fragment, and the cluster cluster to which the target protein fragment belongs can be determined. The protein language term of the representative structural fragment of the cluster is the protein language term corresponding to the target protein fragment.

[0088] Step 103: determining the protein language sentence corresponding to the target protein based on the protein language words corresponding to each target protein fragment.

[0089] Since each target protein fragment is a constituent fragment of the target protein, that is, each target protein fragment constitutes the target protein, the protein language sentence corresponding to the target protein can be determined based on the protein language words corresponding to each target protein fragment, and the protein language sentence includes the protein language words corresponding to each target protein fragment.

[0090] In one possible implementation, the electronic device can determine the position of each target protein fragment in the protein sequence of the target protein; based on the position of each target protein fragment in the protein sequence of the target protein and the protein language words corresponding to each target protein fragment, determine the protein language sentence corresponding to the target protein.

[0091] In one embodiment, the electronic device can sort each target protein fragment based on the position of each target protein fragment in the protein sequence of the target protein, and on this basis, perform feature extraction on each target protein fragment and perform a vocabulary query to obtain the protein language words corresponding to each target protein fragment. The arrangement order of each target protein fragment is the arrangement order of the corresponding protein language words. After the protein language words are arranged, the protein language sentence corresponding to the target protein can be obtained.

[0092] In another embodiment, after determining the position of each target protein fragment in the protein sequence of the target protein, the electronic device extracts features of each target protein fragment and performs a vocabulary query on this basis to obtain protein language words corresponding to each target protein fragment. Based on the position of each target protein fragment in the protein sequence of the target protein, the arrangement order of the protein language words corresponding to each target protein fragment can be determined. After arranging the protein language words, the protein language sentence corresponding to the target protein can be obtained.

[0093] like Figure 4 Shown is a schematic diagram of the conversion of proteins into protein language statements. Figure 4 The protein structure in is the target protein in this embodiment. The electronic device uses the protein structure segmentation method to segment the target protein to obtain the target protein fragment (i.e. Figure 4 The local structural fragments in the target protein are arranged (i.e. Figure 4 ) and extract features for each target protein fragment (i.e. Figure 4 The fragment feature extraction in ) obtains the multi-source heterogeneous features of each target protein fragment (i.e. Figure 4 Based on the extracted multi-source heterogeneous features, a vocabulary query can be performed to query the corresponding protein language terms from the protein language vocabulary (i.e. Figure 4 Based on these coarse-grained language words and the above word order, the protein language sentence (i.e., coarse-grained language sentence) corresponding to the target protein can be obtained. Since each coarse-grained language word can correspond to a natural language word, the coarse-grained language sentence can correspond to a natural language sentence.

[0094] Different from the fine-grained protein language using amino acids as words used in related technologies, the embodiments of the present application provide a method for constructing a coarse-grained protein language. The embodiments of the present application divide proteins into coarse-grained fragments according to structural information, and use machine learning algorithms to summarize a certain number of local structural patterns from a large number of protein fragments based on the physicochemical properties and structural characteristics of the fragments, and construct a coarse-grained vocabulary (i.e., protein language vocabulary) based on this. Then, based on the sequence information (i.e., word order arrangement) and the constructed vocabulary, the protein is converted into a coarse-grained language that is more suitable for molecular-level functional analysis.

[0095] Assume that the protein sequence of a protein is "EVLFQG". After the structural segmentation processing of the embodiment of the present application, the protein fragments "EVLF" and "QG" are mapped to two words "002" and "104" respectively. The coarse-grained sentence corresponding to the protein is "002 104" with a length of 2. The sentence length of the protein language constructed based on the related technology with amino acids as words is 6. Obviously, the protein sentences obtained based on the coarse-grained protein language proposed in the embodiment of the present application are often shorter, which is more conducive to the protein language model to capture the semantic information of the protein molecular level. In addition, coarse-grained words based on structural information (i.e., coarse-grained protein language words) can better characterize the local conserved domain of proteins, thereby helping the protein language model to learn the features at the molecular level and ultimately obtain better protein representation.

[0096] In order to verify the effectiveness of the embodiment of the present application, a data set containing 100,000 proteins can be constructed based on PDB. Based on the above method, these proteins are converted into coarse-grained protein sentences, thereby constructing a corpus to train the protein language model. Based on the above two clustering methods, two protein corpora C k-means and C vq-vae , and trained two protein language models, Doc2Vec model and Bert model. As a control group, a sequence corpus C was constructed using the protein sequence language with amino acids as words. aa , and trained the Doc2Vec model and Bert model with the same parameters.

[0097] The effects of different methods on protein characterization can be evaluated based on the protein function prediction task. Specifically, it is to predict the Gene Ontology (GO) functional label of a given protein, including three subtasks: molecular function (MF) prediction, biological process (BP) prediction and cellular component (CC) prediction. A multi-layer perceptron was used to construct a GO label predictor, and the cross entropy loss was used for model training. The experimental results are listed in Table 1. The evaluation index is the Fmax score. The larger the Fmax score, the better the performance.

[0098] Table 1. GO experimental results of this application and amino acid sequence language on different language models

[0099]

[0100] Among them, C in Table 1 k-means represents the protein corpus obtained by this application based on K-means, C vq-vaeRepresents the protein corpus obtained by this application based on VQ-VAE.

[0101] As shown in Table 1, compared with the amino acid fine-grained language, the coarse-grained structural language proposed in this application has obtained higher Fmax scores in all subtasks of protein function prediction, indicating that the protein coarse-grained language proposed in this application helps the protein language model achieve better protein representation effect. This advantage is particularly prominent in the molecular function (GO-MF) task. vq-vae The Doc2Vec model trained on the coarse-grained language corpus (i.e., the protein corpus obtained based on VQ-VAE) is more accurate than the Doc2Vec model trained on the amino acid sequence language corpus C. aa The Fmax score of the trained Doc2Vec model increased by nearly 30%. k-means The Bert model trained with a coarse-grained language corpus (i.e., a protein corpus obtained based on K-means) is more efficient than the one trained with an amino acid sequence language corpus C. aa The Fmax score of the trained Bert model increased by nearly 29%. This result confirms the above relationship between protein function and local conserved domains, and shows the advantages of the coarse-grained protein language constructed in this application over the fine-grained language based on amino acid terms in protein function prediction.

[0102] Comparing the two vocabulary construction methods, K-means and VQ-VAE each show different advantages. When trained with the Bert language model, the K-means-based method showed an overall higher Fmax score, especially in the GO-MF and GO-CC subtasks. When trained with the Doc2Vec model, the VQ-VAE-based method performed better in the GO-BP task and the GO-MF task. Of the two methods, the K-means-based method has a simpler training process, VQ-VAE is more stable, and is a deep learning model that is expected to achieve better performance under large-scale data-driven conditions. In terms of models, Bert is generally superior to Doc2Vec and is a better choice for building protein language models.

[0103] In general, regardless of whether K-means or VQ-VAE is used to construct the vocabulary, the coarse-grained protein language provided in this application is superior to the fine-grained language based on amino acid terms, providing a new and more effective language definition idea for the development of protein language models.

[0104] The embodiment of the present application divides the target protein into two or more coarse-grained target protein fragments based on secondary structure information, so that the protein language sentences obtained based on the coarse-grained target protein fragments are shorter, which is more conducive to the protein language model to capture the semantic information of the protein molecular level, and the coarse-grained protein language words can better characterize the local conserved domains of the target protein, thereby helping the protein language model to learn the features at the molecular level and ultimately obtain better protein representation.

[0105] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0106] Corresponding to the protein language construction method described in the above embodiment, Figure 5 A schematic diagram of the structure of a protein language construction device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0107] Reference Figure 5 , the device comprises:

[0108] A target segmentation module 51 is used to segment the target protein based on the secondary structure information to obtain two or more target protein fragments;

[0109] A word determination module 52 is used to determine the protein language words corresponding to each target protein fragment;

[0110] The sentence determination module 53 is used to determine the protein language sentence corresponding to the target protein based on the protein language words corresponding to the target protein fragments.

[0111] Optionally, the target segmentation module 51 is specifically used for:

[0112] The amino acids belonging to the same secondary structure information in the protein sequence of the target protein are divided into a target protein fragment.

[0113] Optionally, the word determination module 52 includes:

[0114] A first extraction unit is used to extract the multi-source heterogeneous characteristics of each target protein fragment;

[0115] The word determination unit is used to determine the protein language words corresponding to each target protein fragment from a protein language vocabulary based on the multi-source heterogeneous characteristics of each target protein fragment.

[0116] Optionally, the first extraction unit is specifically used for:

[0117] Extracting amino acid composition characteristics of the target protein fragment;

[0118] Determining the evolutionary information spectrum of the target protein, and extracting the evolutionary conservative features of the target protein fragment from the evolutionary information spectrum of the target protein based on the evolutionary information spectrum of the target protein and the position of the target protein fragment in the protein sequence of the target protein;

[0119] Extracting spatial geometric features of the target protein fragment;

[0120] The amino acid composition characteristics, evolutionary conservation characteristics and spatial geometric characteristics of the target protein fragment are spliced ​​to obtain the multi-source heterogeneous characteristics of the target protein fragment.

[0121] Optionally, the apparatus further includes a vocabulary building module, and the vocabulary building module includes:

[0122] A sample segmentation unit, configured to segment each protein sample based on the secondary structure information to obtain two or more protein sample fragments corresponding to each protein sample;

[0123] The second extraction unit is used to extract the multi-source heterogeneous features of each protein sample fragment;

[0124] A fragment clustering unit, used for clustering the protein sample fragments based on the multi-source heterogeneous characteristics of the protein sample fragments to obtain K clusters, where K is an integer greater than 1;

[0125] An index determination unit, used to determine that the cluster indexes of the K clusters are protein language words corresponding to representative structural fragments of the clusters, and the representative structural fragments of the clusters are cluster centers of the clusters;

[0126] The vocabulary building unit is used to build the protein language vocabulary based on the protein language words of the representative structural fragments of the K clusters.

[0127] Optionally, the second extraction unit is specifically used for:

[0128] Extracting amino acid composition characteristics of the protein sample fragment;

[0129] Determining the evolutionary information spectrum of the protein sample, and extracting the evolutionary conservative features of the protein sample fragment from the evolutionary information spectrum of the protein sample based on the evolutionary information spectrum of the protein sample and the position of the protein sample fragment in the protein sequence of the protein sample;

[0130] Extracting spatial geometric features of the protein sample fragments;

[0131] The amino acid composition characteristics, evolutionary conservation characteristics and spatial geometric characteristics of the protein sample fragments are spliced ​​to obtain the multi-source heterogeneous characteristics of the protein sample fragments.

[0132] Optionally, the sample segmentation unit is specifically used for:

[0133] The amino acids belonging to the same secondary structure information in the protein sequence of the protein sample are divided into a protein sample fragment.

[0134] Optionally, the word determination unit is specifically used for:

[0135] Based on the multi-source heterogeneous characteristics of each target protein fragment, determining the cluster to which each target protein fragment belongs;

[0136] From the protein language vocabulary, the protein language words of the representative structural fragments of the clusters to which the target protein fragments belong are determined, and the protein language words are determined to be the protein language words corresponding to the target protein fragments.

[0137] Optionally, the above device further comprises:

[0138] A position determination module, used to determine the position of each target protein fragment in the protein sequence of the target protein;

[0139] The above statement determination module 53 is specifically used for:

[0140] Based on the position of each target protein fragment in the protein sequence of the target protein and the protein language words corresponding to each target protein fragment, the protein language sentence corresponding to the target protein is determined.

[0141] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0142] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown in the figure), a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, wherein the processor 60 implements the steps of any of the above-mentioned method embodiments when executing the computer program 62.

[0143] The electronic device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 6 and does not constitute a limitation on the electronic device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0144] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0145] In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. In other embodiments, the memory 61 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 6. Further, the memory 61 may also include both an internal storage unit of the electronic device 6 and an external storage device. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0146] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0147] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0148] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0149] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0150] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0151] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for constructing a protein language, characterized in that: include: The target protein is segmented based on the secondary structure information to obtain two or more target protein fragments; Determine the protein language words corresponding to each target protein fragment; Based on the protein language words corresponding to the target protein fragments, a protein language sentence corresponding to the target protein is determined.

2. The method according to claim 1, characterized in that The step of determining the protein language words corresponding to each target protein fragment comprises: Extracting the multi-source heterogeneous features of each target protein fragment; Based on the multi-source heterogeneous characteristics of each target protein fragment, the protein language words corresponding to each target protein fragment are determined from a protein language vocabulary.

3. The method according to claim 2, characterized in that The method for constructing the protein language vocabulary comprises: Segment each protein sample based on the secondary structure information to obtain two or more protein sample fragments corresponding to each protein sample; Extracting multi-source heterogeneous features of each protein sample fragment; Based on the multi-source heterogeneous characteristics of the protein sample fragments, the protein sample fragments are clustered to obtain K clusters, where K is an integer greater than 1; Determine that the cluster indexes of the K clusters are protein language words corresponding to representative structural fragments of the clusters, and the representative structural fragments of the clusters are cluster centers of the clusters; The protein language vocabulary is constructed based on the protein language words of the representative structural fragments of the K clusters.

4. The method according to claim 3, characterized in that The determining of the protein language words corresponding to the target protein fragments from a protein language vocabulary based on the multi-source heterogeneous characteristics of the target protein fragments includes: Based on the multi-source heterogeneous characteristics of each target protein fragment, determining the cluster to which each target protein fragment belongs; From the protein language vocabulary, the protein language words of the representative structural fragments of the clusters to which the target protein fragments belong are determined, and the protein language words are determined to be the protein language words corresponding to the target protein fragments.

5. The method according to claim 2 or 3, characterized in that: Extraction of heterogeneous features of protein fragments, including: extracting amino acid composition characteristics of the protein fragment; Determine the evolutionary information spectrum of the protein corresponding to the protein fragment, and extract the evolutionary conservative features of the protein fragment from the evolutionary information spectrum based on the evolutionary information spectrum and the position of the protein fragment in the protein sequence of the protein; when the protein fragment is the target protein fragment, the protein is the target protein; when the protein fragment is the protein sample fragment, the protein is the protein sample; extracting spatial geometric features of the protein fragment; The amino acid composition feature, the evolutionary conservation feature and the spatial geometric feature are spliced ​​to obtain the multi-source heterogeneous feature of the protein fragment.

6. The method according to claim 1 or 3, characterized in that: The protein is segmented based on the secondary structure information, including: Dividing the amino acids belonging to the same secondary structure information in the protein sequence of the protein into a protein fragment; Wherein, when the protein is the target protein, the protein fragment is the target protein fragment; when the protein is the protein sample, the protein fragment is the protein sample fragment.

7. The method according to any one of claims 1 to 4, characterized in that: Before determining the protein language sentence corresponding to the target protein based on the protein language words corresponding to the target protein fragments, the method further includes: Determining the position of each target protein fragment in the protein sequence of the target protein; The determining of the protein language sentence corresponding to the target protein based on the protein language words corresponding to each target protein fragment comprises: Based on the position of each target protein fragment in the protein sequence of the target protein and the protein language words corresponding to each target protein fragment, the protein language sentence corresponding to the target protein is determined.

8. A protein language construction device, characterized in that: include: A target segmentation module is used to segment the target protein based on the secondary structure information to obtain two or more target protein fragments; A word determination module, used to determine the protein language words corresponding to each target protein fragment; The sentence determination module is used to determine the protein language sentence corresponding to the target protein based on the protein language words corresponding to each target protein fragment.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed, causes the method according to any one of claims 1 to 7 to be performed.

Images