Protein language construction method, device, electronic device and computer program product

Abstract

This application relates to the field of computer technology and provides a method, apparatus, electronic device, and computer program product for constructing a protein language. The method includes: segmenting a target protein based on secondary structure information to obtain two or more target protein fragments; determining the protein language terms corresponding to each target protein fragment; and determining the protein language statement corresponding to the target protein based on the protein language terms corresponding to each target protein fragment. This application facilitates the capture of semantic information at the molecular level by the protein language model, thereby obtaining better protein representation.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a protein language construction method, apparatus, electronic device and computer program product. Background Technology

[0002] Proteins are biological macromolecules composed of 20 standard amino acids. The complexity of an organism is reflected in the number of genes and the number of proteins involved in life activities. Static gene sequences are insufficient to fully reveal the essential laws of life and the occurrence and development of diseases. As a gene product, proteins possess both dynamic and spatiotemporal characteristics, participating in numerous cellular activities, including substance transport, energy metabolism, and immune catalysis. Research on protein function not only promotes the development of life sciences but also helps researchers more accurately determine the causes of diseases, identify corresponding therapeutic targets, and accelerate the development of new drugs.

[0003] Traditional protein research relies on experimental methods, such as using cryo-electron microscopy to resolve the three-dimensional structure of proteins and using wet bioreactors to detect protein function. However, these methods are often time-consuming and costly, and cannot be used for large-scale data analysis. In contrast, computational methods are more efficient and less expensive, and have become an important supplement to wet bioreactors.

[0004] Currently, most machine learning models and mathematical statistics methods cannot handle raw protein sequences and structures. Therefore, for computationally-based protein research, protein representation is a fundamental and crucial step, directly impacting the performance of downstream protein-related tasks. Many current protein representation learning methods treat protein sequences as a language, employing natural language processing (NLP) techniques and using protein language models to learn protein representations. In recent years, researchers have begun using protein language models to learn the intrinsic patterns of proteins, achieving better representation results than traditional methods. Using language models to learn protein representations first requires converting proteins into a language. Currently, most research treats amino acids (AA) as words and amino acid sequences as sentences to define protein language.

[0005] While this fine-grained protein language representation learning model has achieved better representation results than traditional experience-based methods, using only amino acid sequences as the information carried by protein language is very limited. In biological activities, protein structure often determines its function, and structural information plays a crucial role in learning the intrinsic characteristics of proteins. Existing research has demonstrated that protein structure-based representation methods outperform sequence-based methods with less data.

[0006] In response, some studies have attempted to design protein languages ​​that incorporate structural information. This approach combines local residue conformations with amino acids as words, constructing a protein language that integrates sequences and structures. However, similar to the fine-grained protein sequence languages ​​that use amino acids as words, this language remains limited to the amino acid level, hindering protein language models from capturing semantic information and failing to achieve better protein representations. Summary of the Invention

[0007] This application provides a protein language construction method, apparatus, electronic device, and computer program product, which are more conducive to protein language models capturing semantic information at the protein molecular level, thereby obtaining better protein representation.

[0008] In a first aspect, embodiments of this application provide a method for constructing a protein language, including:

[0009] The target protein is segmented based on secondary structure information to obtain two or more target protein fragments;

[0010] Identify the protein language terms corresponding to each target protein fragment;

[0011] Based on the protein language terms corresponding to each target protein fragment, the protein language statement corresponding to the target protein is determined.

[0012] In this embodiment, by segmenting the target protein based on secondary structure information, two or more target protein fragments can be obtained. By determining the protein language terms corresponding to each target protein fragment, the corresponding protein language statement for the target protein can be determined based on the protein language terms corresponding to each target protein fragment, thereby realizing the construction of protein language. Compared with fine-grained protein language using amino acid terms, this scheme segments the target protein into two or more coarse-grained target protein fragments based on secondary structure information. This results in shorter protein language statements obtained from the coarse-grained target protein fragments, which is more conducive to the protein language model capturing semantic information at the molecular level. Furthermore, the coarse-grained protein language terms can better characterize the local conserved domains of the target protein, thereby helping the protein language model learn molecular-level features and ultimately obtain better protein representation.

[0013] Secondly, embodiments of this application provide a protein language construction apparatus, comprising:

[0014] The target segmentation module is used to segment the target protein based on secondary structure information to obtain two or more target protein fragments;

[0015] The word identification module is used to identify the protein language words corresponding to each target protein fragment;

[0016] The statement determination module is used to determine the protein language statement corresponding to the target protein based on the protein language words corresponding to each target protein fragment.

[0017] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device performs the method described in the first aspect above.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a computer, implements the method described in the first aspect above.

[0019] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when run, causes the method described in the first aspect above to be executed.

[0020] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the protein language construction method provided in the embodiments of this application;

[0023] Figure 2 This is a flowchart illustrating the method for determining protein language words provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram illustrating the construction of a protein language vocabulary;

[0025] Figure 4 This is a diagram illustrating the transformation of proteins into protein-language statements;

[0026] Figure 5 This is a schematic diagram of the protein language construction device provided in the embodiments of this application;

[0027] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0028] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0029] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0030] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0031] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0033] Since protein function is often associated with locally conserved domains, these regions maintain high structural similarity or almost no change in the structure of similar proteins. These locally conserved domains may be active sites, binding sites, or regulatory regions of a protein, and are crucial for maintaining protein function. However, fine-grained language, using individual amino acids as terms, has limitations in expressing these locally conserved domains, making it difficult for language models to capture this feature semantically. To better characterize the semantics at the protein molecular level and facilitate downstream protein function analysis, this application proposes a coarse-grained protein language definition strategy for protein function analysis, taking into account the characteristics of locally conserved domains in protein structure. The advantages of this coarse-grained language over fine-grained amino acid language are demonstrated, as detailed in Table 1 below.

[0034] The protein language construction method provided in this application can be applied to electronic devices such as mobile phones, tablets, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device.

[0035] See also Figure 1 , Figure 1 The illustration shows a flowchart of a protein language construction method provided in an embodiment of this application, applied to an electronic device. As an example and not a limitation, the method includes the following steps:

[0036] Step 101: Based on the secondary structure information, the target protein is segmented to obtain two or more target protein fragments.

[0037] The secondary structure information is divided into 8 types, namely helix (H), bridge (B), extended chain (E), 3-helix (G), 5-helix (I), hydrogen bond turn (T), coil (S), ring or random coil (officially marked as blank, but can be customized in this application, for example, defined as C).

[0038] In one embodiment, a Dictionary of Secondary Structure of Protein (DSSP) tool can be used to extract the secondary structure information of a protein.

[0039] For the target protein and the protein samples listed below, the proteins are segmented based on secondary structure information, including:

[0040] The amino acids in the protein sequence that share the same secondary structure information are cut into a protein fragment;

[0041] In the case of a protein being the target protein, the protein fragment is the target protein fragment; in the case of a protein sample, the protein fragment is the protein sample fragment described below.

[0042] This embodiment segments proteins based on secondary structure information, resulting in discrete protein fragments. Since most amino acids in a protein are arranged in a specific spatial structure, each protein fragment contains multiple amino acids. Therefore, the protein language obtained through this segmentation method is coarse-grained. Coarse-grained terms significantly reduce the length of protein sentences, thereby facilitating the protein language model's ability to capture contextual information and learn protein spatial semantics.

[0043] Statistical studies have revealed that the number of fine fragments containing very few amino acids obtained from protein segmentation is relatively small, and the number of excessively long fragments is also extremely rare. Furthermore, fine fragments containing very few amino acids often lack sufficient biological significance and may not provide enough information to describe the protein's function or structure. Excessively long fragments occur infrequently; a moderate fragment length ensures that each protein fragment contains sufficient information to provide specific structures, functions, or patterns, helping the protein language model effectively learn useful features without being interfered with by irrelevant noise or redundant information. Therefore, to minimize the impact of noisy fragments on the subsequent learning performance of the protein language model, in one embodiment, the length of each target protein fragment obtained after segmentation can be determined, and target protein fragments with a length less than a first length threshold or a length greater than a second length threshold can be filtered. Based on target protein fragments with a length greater than or equal to the first length threshold and a length less than or equal to the second length threshold, step 102 (i.e., determining the protein language terms for target protein fragments with a length greater than or equal to the first length threshold and a length less than or equal to the second length threshold) is executed. Here, the length of the target protein fragment can refer to the number of amino acids in the target protein fragment. The first length threshold is less than the second length threshold. The first length threshold mentioned above is used to screen for small fragments containing very few amino acids. The second length threshold mentioned above is used to screen for protein fragments that are too long.

[0044] Step 102: Determine the protein language terms corresponding to each target protein fragment.

[0045] Since the target protein fragment is a coarse-grained protein fragment, the protein language terms corresponding to the target protein fragment are also coarse-grained. In this embodiment, coarse-grained language terms based on secondary structure information can better characterize the local conserved domains of proteins, thereby helping the protein language model learn molecular-level features and ultimately obtain better protein representations.

[0046] See also Figure 2 The electronic device can determine the protein language words according to steps 201 and 202.

[0047] Step 201: Extract the multi-source isomorphic features of each target protein fragment.

[0048] The aforementioned multi-source heterogeneous features are typically composed of features from multiple different sources. Optionally, these multiple features from different sources include, but are not limited to, amino acid composition features, evolutionarily conserved features, and spatial geometric features.

[0049] In one possible implementation, for the target protein fragment and the protein sample fragments described below, the multi-source heterogeneity features of the protein fragments are extracted, including:

[0050] Amino acid composition characteristics of extracted protein fragments;

[0051] The evolutionary information spectrum of the protein corresponding to the protein fragment is determined, and the evolutionary conserved features of the protein fragment are extracted from the evolutionary information spectrum based on the evolutionary information spectrum and the position of the protein fragment in the protein sequence; when the protein fragment is the target protein fragment, the protein is the target protein; when the protein fragment is a protein sample fragment, the protein is the protein sample.

[0052] Extracting the spatial geometric features of protein fragments;

[0053] By splicing together amino acid composition features, evolutionary conservation features, and spatial geometric features, the multi-origin heterogeneous features of protein fragments are obtained.

[0054] Since the percentage and quantity of standard amino acids in a protein fragment can intuitively represent the content of standard amino acids in the fragment, and thus characterize the amino acid composition of the protein fragment, the amino acid composition characteristics of the protein fragment can refer to either the percentage of N standard amino acids in the fragment or the quantity of N standard amino acids in the fragment. N is a positive integer. Optionally, some or all of the standard amino acids in the protein can be selected as the N standard amino acids, depending on the needs of the scenario.

[0055] Using the percentage of N standard amino acids in a protein fragment as the amino acid composition characteristic of the protein fragment can avoid normalization and improve data processing efficiency.

[0056] When the number of N standard amino acids in a protein fragment is used as a characteristic of the protein fragment's amino acid composition, in order to improve data processing efficiency, the number of N standard amino acids in the protein fragment can be normalized.

[0057] As an example rather than a limitation, N is 20. The 20 standard amino acids include glycine, alanine, valine, leucine, isoleucine, methionine, proline, tryptophan, serine, tyrosine, cysteine, phenylalanine, asparagine, glutamine, threonine, aspartic acid, glutamic acid, lysine, arginine, and histidine.

[0058] As an example, and not a limitation, for a given protein fragment containing n amino acids, the corresponding amino acid composition feature is represented as a vector v consisting of 20 elements. a =[e1,e2,…,e 20 The percentages of each of the 20 standard amino acids in the protein fragment are indicated. i =f(P,A i ) / n,f(P,A i To statistically analyze the i-th amino acid A i Frequency of occurrence in protein fragment P (i.e., the i-th amino acid A) i The number of amino acids in protein fragment P is a function of n, where n is the length of the protein fragment (i.e., the number of amino acids in the protein fragment).

[0059] In one embodiment, two types of evolutionary information profiles for proteins can be used: a Position-Specific Scoring Matrix (PSSM) profile and a Hidden Markov Model (HMM) profile. The PSSM profile can be generated by PSI-BLAST based on a protein database specified by a protein search. The HMM profile can be constructed by HHblits based on a protein database specified by a protein search.

[0060] It should be noted that since the final form of the PSSM spectrum and HMM spectrum is a matrix, the PSSM spectrum can be called the PSSM matrix and the HMM spectrum can be called the HMM matrix.

[0061] Evolutionarily conserved features of protein fragments can be extracted from the evolutionary information spectrum at the corresponding positions. For example, if a protein fragment is located at [5, 10] in the protein sequence, meaning that amino acids 5 through 10 belong to this protein fragment, then information from lines 5 through 10 should be extracted from the protein's evolutionary information spectrum. Before feature extraction, the PSSM and HMM spectra need to be normalized to reduce features with large numerical ranges to a suitable range, thus controlling the data scale. For HMM spectra, the elements can be normalized as follows:

[0062]

[0063] Where x1 represents an element in the HMM spectrum. This represents the normalized value of the element, with * indicating infinity. If element x1 in the HMM spectrum is infinity, then the normalized value of this element is 1; if element x1 in the HMM spectrum is not infinity, then the normalized value is 1.

[0064] For the PSSM spectrum, the elements can be normalized using the sigmoid function, as shown in the following formula:

[0065]

[0066] Where x2 represents an element in the PSSM spectrum, This represents the normalized value of the element.

[0067] Both evolutionary conserved features based on PSSM and HMM spectra can be represented as an n*20 matrix, where n is the number of amino acids in the protein fragment. To facilitate subsequent feature concatenation, the two evolutionary conserved features of the protein fragment are dimensionality-reduced by summing the information in each column and dividing by the length of the protein fragment, resulting in two 20-dimensional vectors. Concatenating these two 20-dimensional vectors yields the evolutionary conserved features of the protein fragment.

[0068] This embodiment extracts spatial geometric information from protein fragments, which can enrich the structural semantics of protein language. Currently, spatial geometric information includes the secondary structure information of structural fragments (i.e., protein fragments), obtained by DSSP tools based on protein structure analysis. This embodiment uses 8-dimensional one-hot vectors to represent the eight secondary structure types used. Further spatial geometric features can be represented using hemispherical exposure area, residue depth information, etc. Optionally, by extracting residue-level structural information and using sequence models to learn the global structural information of protein fragments, the structural features of protein fragments can be obtained as a supplement to spatial geometric features.

[0069] For each protein fragment, a 68-dimensional feature vector can be obtained by splicing together the amino acid composition features, evolutionary conservation features, and spatial geometric features of the protein fragment.

[0070] Step 202: Based on the multi-source heterogeneity of each target protein fragment, determine the corresponding protein language terms for each target protein fragment from the protein language lexicon.

[0071] For each target protein fragment, the corresponding protein language terms can be found in the protein language lexicon.

[0072] To avoid the problem of an excessively large protein language vocabulary space, in one embodiment, a semantic space clustering strategy can be used to construct a protein language vocabulary of appropriate size. Then, the words corresponding to protein structural fragments in the protein language vocabulary can be determined through semantic space mapping, and finally, protein sentences can be obtained by arranging the words in the order of the words.

[0073] In one possible implementation, the protein language vocabulary can be constructed in the following ways:

[0074] Based on secondary structure information, each protein sample is segmented to obtain two or more protein sample fragments corresponding to each protein sample.

[0075] Extract the multi-source isomorphic features of each protein sample fragment;

[0076] Based on the multi-source heterogeneity of each protein sample fragment, the protein sample fragments are clustered to obtain K clusters, where K is an integer greater than 1;

[0077] The cluster index of K clusters is determined as the protein language word of the representative structural fragment of the corresponding cluster, and the representative structural fragment of the cluster is the cluster center;

[0078] A protein language vocabulary is constructed based on representative structural fragments of K clusters.

[0079] The protein samples mentioned above refer to proteins used to construct a protein language vocabulary.

[0080] K represents the number of protein language terms in the protein language lexicon, i.e., the vocabulary size. Protein fragments within a cluster share similar characteristics; therefore, all protein fragments within a cluster correspond to the same protein language term. In other words, one cluster can represent one protein language term, which can be identified using a cluster index. For example, if K is 800, there are 800 clusters with cluster indices ranging from 000 to 799. If a protein fragment is clustered into the cluster with index 008, then 008 can represent that protein fragment; in this case, 008 is the protein language term for that protein fragment.

[0081] This embodiment employs an unsupervised clustering method to summarize protein sample fragments, clustering all the protein sample fragment representations obtained above. Each cluster (i.e., the cluster index) is treated as a word, thereby constructing a protein structural language vocabulary of a specified lexical space. Protein sample fragments belonging to a particular cluster are represented by their corresponding words (i.e., the cluster index), and the number of clusters is the size of the constructed protein language vocabulary.

[0082] This embodiment can cluster protein sample fragments using two methods: 1) clustering using the classic K-means algorithm; 2) clustering using the Vector Quantized Variational Autoencoder (VQ-VAE).

[0083] K-means is a classic unsupervised clustering method that can cluster protein sample fragments into K clusters. For example, in this embodiment, the value of K is set to 800, which means that the size K of the protein language vocabulary is 800.

[0084] VQ-VAE is a generative model that combines variational autoencoders and vector quantization techniques to learn discrete latent representations. Primarily used in generative domains, this model maps the latent representation of a sample to its nearest-neighbor vectors in the codebook during the encoding phase, achieving an implicit clustering effect and thus serving as a clustering method. During training, VQ-VAE adaptively adjusts the encoder and codebook so that the vectors fit the latent space representation, thereby achieving discrete clustering of protein sample fragments. The codebook size is customizable; for example, in this embodiment, the codebook size is set to 1000, meaning the protein language vocabulary size K is 1000.

[0085] After the protein language vocabulary is constructed, the electronic device can convert the target protein into coarse-grained language sentences. For example... Figure 3 The diagram shown illustrates the construction of a protein language vocabulary. Figure 3 The protein structure dataset (PDB) is used to provide protein samples; after obtaining each protein sample from the protein structure dataset, the electronic device uses a protein structure segmentation method to segment each protein sample, obtaining protein sample fragments for each protein sample; feature extraction is performed on each protein sample fragment (i.e., Figure 3 (Fragment feature extraction) to obtain the multi-source heterogeneous features of each protein sample fragment (i.e., Figure 3 Local structural fragment features in the protein samples); based on multi-source heterogeneity features and clustering methods, each protein sample fragment is clustered to obtain K clusters (i.e., Figure 3 The clustering space in the protein language and the protein language words of representative structural fragments constitute the protein language vocabulary (i.e., the protein language lexicon). Figure 3 (A coarse-grained language vocabulary). The aforementioned protein structure dataset can also be referred to as a protein structure database. The protein structure segmentation method described above is the method for segmenting proteins based on secondary structure information provided in this embodiment.

[0086] In one possible implementation, when determining the protein language words corresponding to each target protein fragment from the protein language lexicon, the electronic device can determine the cluster to which each target protein fragment belongs based on the multi-source heterogeneity characteristics of each target protein fragment; from the protein language lexicon, determine the protein language words of the representative structural fragments of the cluster to which each target protein fragment belongs, and determine that the protein language words are the protein language words corresponding to the target protein fragment.

[0087] Since the protein language vocabulary is obtained by clustering based on the multi-source heterogeneity features of protein sample fragments, for each target protein fragment, the target protein fragment can be clustered in the existing clustering space (i.e., clusters) based on its multi-source heterogeneity features. The cluster to which the target protein fragment belongs is then determined, and the protein language words of the representative structural fragments of the cluster are the protein language words corresponding to the target protein fragment.

[0088] Step 103: Based on the protein language words corresponding to each target protein fragment, determine the protein language statement corresponding to the target protein.

[0089] Since each target protein fragment is a component of the target protein, meaning that each target protein fragment constitutes the target protein, the protein language statement corresponding to the target protein can be determined based on the protein language terms corresponding to each target protein fragment. This protein language statement includes the protein language terms corresponding to each target protein fragment.

[0090] In one possible implementation, the electronic device can determine the position of each target protein fragment in the protein sequence of the target protein; based on the position of each target protein fragment in the protein sequence of the target protein and the protein language words corresponding to each target protein fragment, it can determine the protein language statement corresponding to the target protein.

[0091] In one embodiment, the electronic device can sort the target protein fragments based on their positions in the protein sequence of the target protein. On this basis, feature extraction and word lookup are performed on each target protein fragment to obtain the protein language words corresponding to each target protein fragment. The order of the target protein fragments is the order of the corresponding protein language words. After arranging the protein language words, the protein language statement corresponding to the target protein can be obtained.

[0092] In another embodiment, after determining the position of each target protein fragment in the protein sequence of the target protein, the electronic device performs feature extraction and word lookup on each target protein fragment to obtain the protein language words corresponding to each target protein fragment. Based on the position of each target protein fragment in the protein sequence of the target protein, the order of the protein language words corresponding to each target protein fragment can be determined. After arranging the protein language words, the protein language statement corresponding to the target protein can be obtained.

[0093] like Figure 4 The diagram shows how proteins are translated into protein language statements. Figure 4 The protein structure shown is the target protein in this embodiment. The electronic device uses a protein structure segmentation method to segment the target protein, obtaining the target protein fragment (i.e., Figure 4 Local structural fragments in the protein), and arrange the target protein fragments (i.e., Figure 4 (Word order arrangement); feature extraction for each target protein fragment (i.e.) Figure 4 (Fragment feature extraction) to obtain the multi-source heterogeneous features of each target protein fragment (i.e., Figure 4 Local structural fragment features in the protein language); based on the extracted multi-source heterogeneous features, a lexicon query can be performed to retrieve the corresponding protein language words from the protein language lexicon (i.e., Figure 4 Based on the coarse-grained language words in the text and their arrangement in the above order, the protein language statement (i.e., the coarse-grained language statement) corresponding to the target protein can be obtained. Since each coarse-grained language word can correspond to one natural language word, each coarse-grained language statement can correspond to one natural language statement.

[0094] Unlike related technologies that employ fine-grained protein language based on amino acids, this application provides a method for constructing a coarse-grained protein language. This application segmentes proteins into coarse-grained fragments based on structural information. Based on the physicochemical properties and structural features of these fragments, machine learning algorithms are used to summarize a number of local structural patterns from a large number of protein fragments, thereby constructing a coarse-grained vocabulary (i.e., a protein language vocabulary). Then, based on sequence information (i.e., word order) and the constructed vocabulary, the protein is transformed into a coarse-grained language more suitable for molecular-level functional analysis.

[0095] Suppose a protein sequence is “EVLFQG”. After structural segmentation processing according to the embodiments of this application, the protein fragments “EVLF” and “QG” are mapped to the words “002” and “104”, respectively. The corresponding coarse-grained sentence for this protein is then “002 104”, with a length of 2. In contrast, sentences constructed using amino acid terms in protein language based on related technologies have a length of 6. Clearly, protein sentences obtained using the coarse-grained protein language proposed in the embodiments of this application are often shorter, which is more conducive to protein language models capturing semantic information at the molecular level. Furthermore, coarse-grained terms based on structural information (i.e., coarse-grained protein language terms) can better characterize local conserved domains of proteins, thereby helping protein language models learn molecular-level features and ultimately obtain better protein representations.

[0096] To verify the effectiveness of the embodiments of this application, a dataset containing 100,000 proteins can be constructed based on PDB. These proteins are then converted into coarse-grained protein sentences using the methods described above, thereby constructing a corpus to train a protein language model. Two protein corpora, C, are obtained based on the two clustering methods described above. k-means and C vq-vae Two protein language models, Doc2Vec and Bert, were trained. As a control group, a sequence corpus C was constructed using protein sequence language with amino acid terms. aa They also trained Doc2Vec and Bert models with the same parameters.

[0097] The effectiveness of different methods for protein characterization can be evaluated based on protein function prediction tasks. Specifically, this involves predicting the gene ontology (GO) functional tags of a given protein, including three sub-tasks: molecular function (MF) prediction, biological process (BP) prediction, and cellular component (CC) prediction. A multilayer perceptron was used to construct the GO tag predictor, and cross-entropy loss was employed for model training. Experimental results are listed in Table 1, with the Fmax score serving as the evaluation metric; a higher Fmax score indicates better performance.

[0098] Table 1. GO experimental results of this application and amino acid sequence language on different language models.

[0099]

[0100] Among them, C in Table 1 k-means This application represents a protein corpus obtained based on K-means, C vq-vaeThis application indicates that the protein corpus obtained is based on VQ-VAE.

[0101] As shown in Table 1, compared with the fine-grained amino acid language, the coarse-grained structural language proposed in this application achieved higher Fmax scores in all sub-tasks of protein function prediction, indicating that the proposed coarse-grained protein language helps the protein language model achieve better protein representation. This advantage is particularly prominent in the molecular function (GO-MF) task. (Using C...) vq-vae The Doc2Vec model trained on a coarse-grained language corpus (i.e., a protein corpus obtained based on VQ-VAE) is superior to the model trained on an amino acid sequence language corpus. aa The trained Doc2Vec model improved its Fmax score by nearly 30%. (Using C...) k-means The BERT model trained on a coarse-grained language corpus (i.e., a protein corpus obtained based on K-means) is superior to the model trained on an amino acid sequence language corpus. aa The trained BERT model showed an improvement of nearly 29% in its Fmax score. This result confirms the above-mentioned relationship between protein function and local conserved domains, demonstrating the advantage of the coarse-grained protein language constructed in this application over fine-grained language based on amino acid terms in protein function prediction.

[0102] Comparing the two vocabulary construction methods, K-means and VQ-VAE each demonstrate different advantages. When trained using the BERT language model, the K-means-based method exhibits a higher overall Fmax score, especially in the GO-MF and GO-CC subtasks. When trained using the Doc2Vec model, the VQ-VAE-based method performs better in the GO-BP and GO-MF tasks. Of the two methods, the K-means-based method is simpler to train, while VQ-VAE is more stable and, being a deep learning model, is expected to achieve better performance with large-scale data. In terms of the model itself, BERT is generally superior to Doc2Vec and is the better choice for building protein language models.

[0103] Overall, regardless of whether K-means or VQ-VAE is used to construct the vocabulary, the coarse-grained protein language provided in this application is superior to the fine-grained language based on amino acid terms, offering a novel and more effective approach to language definition for the development of protein language models.

[0104] This application embodiment divides the target protein into two or more coarse-grained target protein fragments based on secondary structure information. This results in shorter protein language statements obtained from the coarse-grained target protein fragments, which is more conducive to the protein language model capturing semantic information at the molecular level. Furthermore, the coarse-grained protein language terms can better characterize the local conserved domains of the target protein, thereby helping the protein language model learn molecular-level features and ultimately obtain better protein representations.

[0105] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0106] Corresponding to the protein language construction method described in the above embodiments, Figure 5 A schematic diagram of the protein language construction apparatus provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0107] Reference Figure 5 The device includes:

[0108] The target segmentation module 51 is used to segment the target protein based on secondary structure information to obtain two or more target protein fragments;

[0109] The word determination module 52 is used to determine the protein language words corresponding to each target protein fragment;

[0110] The statement determination module 53 is used to determine the protein language statement corresponding to the target protein based on the protein language words corresponding to each target protein fragment.

[0111] Optionally, the target segmentation module 51 described above is specifically used for:

[0112] The amino acids in the protein sequence of the target protein that belong to the same secondary structure information are divided into a target protein fragment.

[0113] Optionally, the word determination module 52 mentioned above includes:

[0114] The first extraction unit is used to extract the multi-source heterogeneity features of each target protein fragment;

[0115] The word determination unit is used to determine the protein language words corresponding to each target protein fragment from a protein language lexicon based on the multi-source heterogeneity characteristics of each target protein fragment.

[0116] Optionally, the first extraction unit described above is specifically used for:

[0117] Extract the amino acid composition characteristics of the target protein fragment;

[0118] The evolutionary information spectrum of the target protein is determined, and based on the evolutionary information spectrum of the target protein and the position of the target protein fragment in the protein sequence of the target protein, the evolutionary conserved features of the target protein fragment are extracted from the evolutionary information spectrum of the target protein.

[0119] Extract the spatial geometric features of the target protein fragment;

[0120] The amino acid composition characteristics, evolutionary conservation characteristics, and spatial geometric characteristics of the target protein fragment are spliced ​​together to obtain the multi-source heterogeneity characteristics of the target protein fragment.

[0121] Optionally, the above apparatus further includes a vocabulary building module, which includes:

[0122] The sample segmentation unit is used to segment each protein sample based on the secondary structure information to obtain two or more protein sample fragments corresponding to each protein sample.

[0123] The second extraction unit is used to extract the multi-source heterogeneity features of each protein sample fragment;

[0124] The fragment clustering unit is used to cluster the protein sample fragments based on their multi-source heterogeneity characteristics to obtain K clusters, where K is an integer greater than 1.

[0125] An index determination unit is used to determine the cluster index of the K clusters as a protein language word representing a representative structural fragment of the corresponding cluster, wherein the representative structural fragment of the cluster is the cluster center of the cluster.

[0126] The vocabulary construction unit is used to construct the protein language vocabulary based on the protein language words of representative structural fragments of the K clusters.

[0127] Optionally, the second extraction unit described above is specifically used for:

[0128] Extract the amino acid composition characteristics of the protein sample fragment;

[0129] The evolutionary information spectrum of the protein sample is determined, and based on the evolutionary information spectrum of the protein sample and the position of the protein sample fragment in the protein sequence of the protein sample, the evolutionary conserved features of the protein sample fragment are extracted from the evolutionary information spectrum of the protein sample.

[0130] Extract the spatial geometric features of the protein sample fragment;

[0131] The amino acid composition characteristics, evolutionary conservation characteristics, and spatial geometric characteristics of the protein sample fragments are spliced ​​together to obtain the multi-source heterogeneity characteristics of the protein sample fragments.

[0132] Optionally, the above sample segmentation unit is specifically used for:

[0133] The amino acids in the protein sequence of the protein sample that belong to the same secondary structure information are divided into a protein sample fragment.

[0134] Optionally, the above word determination unit is specifically used for:

[0135] Based on the multi-source isomorphic characteristics of each target protein fragment, the cluster to which each target protein fragment belongs is determined;

[0136] From the protein language lexicon, determine the protein language words of the representative structural fragments of the cluster to which each target protein fragment belongs, and determine that the protein language words are the protein language words corresponding to the target protein fragments.

[0137] Optionally, the above-mentioned device further includes:

[0138] A location determination module is used to determine the location of each target protein fragment within the protein sequence of the target protein;

[0139] The above statement determination module 53 is specifically used for:

[0140] Based on the position of each target protein fragment in the protein sequence of the target protein and the protein language words corresponding to each target protein fragment, the protein language statement corresponding to the target protein is determined.

[0141] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0142] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 6 (Only one is shown in the diagram), memory 61, and computer program 62 stored in said memory 61 and executable on said at least one processor 60, which, when executed, implements the steps in any of the above method embodiments.

[0143] The electronic device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0144] The processor 60 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0145] In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. In other embodiments, the memory 61 may be an external storage device of the electronic device 6, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 6. Furthermore, the memory 61 may include both internal and external storage units of the electronic device 6. The memory 61 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0146] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0147] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0148] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0149] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for constructing a protein language, characterized in that, include: The target protein is segmented based on secondary structure information to obtain two or more target protein fragments; Identify the protein language terms corresponding to each target protein fragment; Based on the protein language words corresponding to each target protein fragment, determine the protein language statement corresponding to the target protein; The process of determining the protein language terms corresponding to each target protein fragment includes: Extract the multi-source isomorphism features of each target protein fragment; Based on the multi-source heterogeneity of each target protein fragment, the corresponding protein language terms for each target protein fragment are determined from a protein language lexicon. The extraction of multi-source isomorphic features of each target protein fragment includes: For any of the target protein fragments, extract the amino acid composition characteristics of the target protein fragments; The evolutionary information spectrum of the target protein is determined, and based on the evolutionary information spectrum of the target protein and the position of the target protein fragment in the protein sequence of the target protein, the evolutionary conserved features of the target protein fragment are extracted from the evolutionary information spectrum of the target protein. Extract the spatial geometric features of the target protein fragment; The amino acid composition features, evolutionary conservation features, and spatial geometric features of the target protein fragment are spliced ​​together to obtain the multi-source heterogeneity features of the target protein fragment; Before determining the protein language statement corresponding to the target protein based on the protein language words corresponding to each target protein fragment, the method further includes: Determine the position of each target protein fragment within the protein sequence of the target protein; The step of determining the protein language statement corresponding to the target protein based on the protein language words corresponding to each target protein fragment includes: Based on the position of each target protein fragment in the protein sequence of the target protein and the protein language words corresponding to each target protein fragment, the protein language statement corresponding to the target protein is determined.

2. The method according to claim 1, characterized in that, The construction method of the protein language vocabulary includes: Based on the secondary structure information, each protein sample is segmented to obtain two or more protein sample fragments corresponding to each protein sample. Extract the multi-source isomorphic features of each protein sample fragment; Based on the multi-source heterogeneity characteristics of each protein sample fragment, the protein sample fragments are clustered to obtain K clusters, where K is an integer greater than 1; The cluster index of the K clusters is determined as a protein language word representing a representative structural fragment of the corresponding cluster, and the representative structural fragment of the cluster is the cluster center of the cluster. The protein language vocabulary is constructed based on the protein language words of representative structural fragments of the K clusters.

3. The method according to claim 2, characterized in that, The process of determining the corresponding protein language terms for each target protein fragment from a protein language lexicon based on the multi-source heterogeneity characteristics of each target protein fragment includes: Based on the multi-source isomorphic characteristics of each target protein fragment, the cluster to which each target protein fragment belongs is determined; From the protein language lexicon, determine the protein language words of the representative structural fragments of the cluster to which each target protein fragment belongs, and determine that the protein language words are the protein language words corresponding to the target protein fragments.

4. The method according to claim 2, characterized in that, The extraction of multi-source heterogeneity features from each protein sample fragment includes: For any of the protein sample fragments, extract the amino acid composition characteristics of the protein sample fragments; The evolutionary information spectrum of the protein sample is determined, and based on the evolutionary information spectrum of the protein sample and the position of the protein sample fragment in the protein sequence of the protein sample, the evolutionary conserved features of the protein sample fragment are extracted from the evolutionary information spectrum of the protein sample. Extract the spatial geometric features of the protein sample fragment; The amino acid composition characteristics, evolutionary conservation characteristics, and spatial geometric characteristics of the protein sample fragments are spliced ​​together to obtain the multi-source heterogeneity characteristics of the protein sample fragments.

5. The method according to claim 1 or 2, characterized in that, Based on the aforementioned secondary structure information, the protein is segmented, including: The amino acids in the protein sequence that belong to the same secondary structure information are divided into a protein fragment; Wherein, if the protein is the target protein, the protein fragment is the target protein fragment; if the protein is the protein sample, the protein fragment is the protein sample fragment.

6. A protein language construction device, characterized in that, include: The target segmentation module is used to segment the target protein based on secondary structure information to obtain two or more target protein fragments; The word identification module is used to identify the protein language words corresponding to each target protein fragment; The statement determination module is used to determine the protein language statement corresponding to the target protein based on the protein language words corresponding to each target protein fragment; The word determination module includes: The first extraction unit is used to extract the multi-source heterogeneity features of each target protein fragment; The word determination unit is used to determine the protein language words corresponding to each target protein fragment from a protein language lexicon based on the multi-source heterogeneity features of each target protein fragment. The first extraction unit is specifically used for: For any of the target protein fragments, extract the amino acid composition characteristics of the target protein fragments; The evolutionary information spectrum of the target protein is determined, and based on the evolutionary information spectrum of the target protein and the position of the target protein fragment in the protein sequence of the target protein, the evolutionary conserved features of the target protein fragment are extracted from the evolutionary information spectrum of the target protein. Extract the spatial geometric features of the target protein fragment; The amino acid composition features, evolutionary conservation features, and spatial geometric features of the target protein fragment are spliced ​​together to obtain the multi-source heterogeneity features of the target protein fragment; The device further includes: A location determination module is used to determine the location of each target protein fragment within the protein sequence of the target protein; The statement determination module is specifically used for: Based on the position of each target protein fragment in the protein sequence of the target protein and the protein language words corresponding to each target protein fragment, the protein language statement corresponding to the target protein is determined.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the method as described in any one of claims 1 to 5.

8. A computer program product, characterized in that, Includes a computer program, which, when run, causes the method as described in any one of claims 1 to 5 to be performed.

Images