Protein representation learning system and method

Abstract

The protein representation learning system and method performed by at least one processor may be configured to learn a protein representation using protein amino acid sequences, the method comprising the steps of: obtaining one or more pieces of protein amino acid sequence information; generating an intermediate representation from the protein amino sequence information using an encoder neural network and generating a projected representation from the intermediate representation using a projection neural network; evaluating a contrastive loss function; and improving one or more numerical values or parameters of one or more of the encoder neural network and the projection neural network, on the basis of the contrastive loss function.

Description

Protein expression learning system and method

[0001] The present disclosure relates to a system and method for predicting the physical properties, structure, binding affinity, and interaction of proteins, or for designing proteins, using data about proteins. More specifically, the present disclosure relates to a system and method for training a neural network to generate protein representations using protein amino acid sequence information.

[0002] Proteins have long been recognized as core molecules of life, underlying various biological phenomena. However, their structure and function remained elusive until recently. With the completion of the Genome Project and rapid advancements in gene sequencing technology, a significant portion of the amino acid sequences that make up proteins in living organisms have been elucidated. However, aspects of protein structure, properties, and function remain incomplete.

[0003] With the recent advancement of artificial intelligence technology, research is being conducted on predicting protein function using protein information. AI models are being used to predict protein properties, structures, and binding affinity for specific molecules from protein data, as well as to design amino acid sequences for proteins with desired properties. For AI models to learn protein amino acid sequence information and apply the predictions they make in real-world research, their reliability must be sufficiently high. Therefore, an AI model capable of producing highly reliable predictions is essential.

[0004] To obtain highly reliable prediction results from data learned by an artificial intelligence model, it is important to utilize an algorithm designed to obtain accurate protein representations from protein amino acid sequence information.

[0005] Conventional algorithms used to obtain protein representations from protein amino acid sequence information often result in performance degradation when used for downstream tasks. Therefore, a protein representation generation system that addresses the shortcomings of conventional algorithms and generates protein representations is needed to improve the performance of AI models.

[0006] Through this disclosure, we aim to provide a system and method capable of generating a protein expression that accurately reflects the characteristics of protein amino acid sequence information, compared to a protein expression generation system that has been used in the past.

[0007] Through this disclosure, it is intended to provide a system and method for predicting the physical properties, structure, binding force, interaction, etc. of a protein from protein amino acid sequence information, and to provide a system and method with higher prediction reliability than the systems used in the past.

[0008] The present disclosure seeks to provide a system and method capable of generating protein expressions that reflect protein interaction partner information.

[0009] Through this disclosure, it is intended to provide a system and method for predicting the physical properties, structure, binding force, interaction, etc. of a protein from data about the protein, or for designing an amino acid sequence of a protein having desired properties, and to provide a system and method with higher prediction reliability than the systems used in the past.

[0010] One embodiment of the present disclosure can provide a system and method for learning protein expression using protein amino acid sequences.

[0011] One embodiment of the present disclosure is directed to providing a system comprising at least one processor and a memory storing one or more instructions.

[0012] The at least one processor learns protein representation using protein amino acid sequences by executing the one or more instructions stored in the memory,

[0013] The above learning is,

[0014] A step of obtaining information on one or more protein amino acid sequences,

[0015] A step of generating a first intermediate representation from the protein amino acid sequence information using an encoder neural network, and generating a first projected representation from the first intermediate representation using a projection neural network;

[0016] A step of generating a plurality of decomposed protein amino acid sequence information by dividing the protein amino acid sequence information, generating a plurality of second intermediate representations from the plurality of decomposed protein amino acid sequence information using the encoder neural network, and then generating an integrated second intermediate representation by integrating at least one of the plurality of second intermediate representations, and then generating a second projected representation using the projection neural network.

[0017] A step of evaluating a contrast loss function from the first projected representation and the second projected representation, and

[0018] The step of improving one or more values ​​or parameters of one or more neural networks among the encoder neural network and the projection neural network may be included based on the contrast loss function.

[0019] In one embodiment, in the step of generating an integrated second intermediate representation by integrating one or more of the plurality of second intermediate representations, the integration may be performed in one method selected from among summing, averaging, and concatenation.

[0020] In one embodiment, the step of dividing the protein amino acid sequence information to generate decomposed protein amino acid sequence information may divide the protein amino acid sequence information into two decomposed protein amino acid sequence information.

[0021] In one embodiment, the encoder neural network may be a transformer encoder model.

[0022] In one embodiment, the projection neural network may be a linear projection model.

[0023] In one embodiment, the protein amino acid sequence information may be amino acid sequence information of an antibody or a fragment thereof.

[0024] In one embodiment, the protein amino acid sequence information may be amino acid sequence information of a complementarity determining region of an antibody or a fragment thereof.

[0025] One embodiment of the present disclosure seeks to provide a protein representation learning method performed by at least one processor.

[0026] The method comprises a step of learning protein expression using a protein amino acid sequence,

[0027] The above learning is,

[0028] A step of obtaining information on one or more protein amino acid sequences,

[0029] A step of generating a first intermediate representation from the protein amino acid sequence information using an encoder neural network, and generating a first projected representation from the first intermediate representation using a projection neural network;

[0030] A step of generating a plurality of decomposed protein amino acid sequence information by decomposing the protein amino acid sequence information, generating a plurality of second intermediate representations from the plurality of decomposed protein amino acid sequence information using the encoder neural network, and then generating an integrated second intermediate representation by integrating at least one of the plurality of second intermediate representations, and then generating a second projected representation through the projection neural network.

[0031] A step of evaluating a contrast loss function from the first projected representation and the second projected representation, and

[0032] The step of improving one or more values ​​or parameters of one or more neural networks among the encoder neural network and the projection neural network may be included based on the contrast loss function.

[0033] In one embodiment, in the step of generating an integrated second intermediate representation by integrating one or more of the plurality of second intermediate representations, the integration may be performed in one method selected from among summing, averaging, and concatenation.

[0034] In one embodiment, the step of dividing the protein amino acid sequence information to generate decomposed protein amino acid sequence information may divide the protein amino acid sequence information into two decomposed protein amino acid sequence information.

[0035] In one embodiment, the encoder neural network may be a transformer encoder model.

[0036] In one embodiment, the projection neural network may be a linear projection model.

[0037] In one embodiment, the protein amino acid sequence information may be amino acid sequence information of an antibody or a fragment thereof.

[0038] In one embodiment, the protein amino acid sequence information may be amino acid sequence information of a complementarity determining region of an antibody or a fragment thereof.

[0039] One embodiment of the present disclosure includes a program stored on a recording medium to cause a computer to execute a method according to one embodiment of the present disclosure.

[0040] One embodiment of the present disclosure includes a program stored on a computer-readable recording medium that causes a computer to execute a method according to one embodiment of the present disclosure.

[0041] One embodiment of the present disclosure may provide a supervised protein expression learning system.

[0042] One embodiment of the present disclosure may provide a system including a memory storing one or more instructions; and at least one processor executing the one or more instructions stored in the memory.

[0043] The actions performed by one or more of the above commands are

[0044] A step of acquiring protein data, wherein the protein data is protein amino acid sequence data labeled with interaction partner information.

[0045] generating protein representations from the above protein data using one or more encoder neural networks, and

[0046] A contrastive learning step for obtaining a contrastive loss from the protein expression and modifying one or more values ​​of one or more parameters of one or more encoder neural networks based on the contrastive loss,

[0047] Here, the above contrastive learning step may be to train the interaction partners to express the distances of similar proteins close to each other.

[0048] In one embodiment, the step of obtaining the contrast loss from the protein expressions may include calculating distances between protein expressions using a distance function and obtaining the contrast loss based on an average of the distances.

[0049] In one embodiment, the distance function may be selected from a Euclidean distance function, a Manhattan distance, a Chebyshev distance, a cosine distance, a Minkowski distance, a Levenshtein distance, a Mahalanobis distance, a Jaccard distance, and a Hamming distance.

[0050] In one embodiment, the step of generating a protein representation from the protein data can be performed by combining multiple encoder neural networks.

[0051] In one embodiment, the plurality of encoder neural networks may include one or more pre-trained protein language models.

[0052] In one embodiment, the protein may be an antibody and the interaction partner may be an antigen.

[0053] In one embodiment, the supervised protein representation learning system may be a supervised protein representation learning system for a protein structure prediction downstream task.

[0054] One embodiment of the present disclosure may provide a supervised protein representation learning method.

[0055] One embodiment of the present disclosure is a supervised protein representation learning method performed by at least one processor, the method comprising a step of learning a protein representation using a protein amino acid sequence,

[0056] The above learning is,

[0057] A step of acquiring protein data, wherein the protein data is protein amino acid sequence data labeled with interaction partner information.

[0058] generating protein representations from the protein data using one or more encoder neural networks; and

[0059] A contrastive learning step for obtaining a contrastive loss from the protein expression and modifying one or more values ​​of one or more parameters of one or more encoder neural networks based on the contrastive loss,

[0060] Here, the above contrastive learning step may be to train the interaction partners to express the distances of similar proteins close to each other.

[0061] In one embodiment, the step of obtaining the contrast loss from the protein expressions may include calculating distances between protein expressions using a distance function and obtaining the contrast loss based on an average of the distances.

[0062] In one embodiment, the distance function may be selected from a Euclidean distance function, a Manhattan distance, a Chebyshev distance, a cosine distance, a Minkowski distance, a Levenshtein distance, a Mahalanobis distance, a Jaccard distance, and a Hamming distance.

[0063] In one embodiment, the step of generating a protein representation from the protein data may be performed by combining multiple encoder neural networks.

[0064] In one embodiment, the plurality of encoder neural networks may include one or more pre-trained protein language models.

[0065] In one embodiment, the protein may be an antibody and the interaction partner may be an antigen.

[0066] In one embodiment, the supervised protein representation learning method may be a supervised protein representation learning method for a downstream task of protein structure prediction.

[0067] One embodiment of the present disclosure can provide a program stored on a computer-readable recording medium to execute the above method on a computer.

[0068] According to one embodiment of the present disclosure, a system and method for learning protein expression from protein amino acid sequence information can be obtained.

[0069] The protein expression learning system and method according to one embodiment of the present disclosure learns protein amino acid sequence information in a self-supervised contrastive learning manner, and thus can perform contrastive learning without labeled learning data, thereby performing protein expression learning economically.

[0070] The protein expression learning system and method according to one embodiment of the present disclosure generates protein amino acid sequence information decomposed from protein amino acid sequence information, which is learning target data, and performs self-supervised contrastive learning, thereby obtaining a protein expression reflecting the semantic composition of the protein amino acid sequence.

[0071] Additionally, according to one embodiment of the present disclosure, a system and method learned to generate protein representations reflecting protein interaction partner information can be obtained.

[0072] According to one embodiment of the present disclosure, a protein representation reflecting protein interaction partner information can be obtained by a learned system and method.

[0073] The protein representation learning system and method according to one embodiment of the present disclosure can be utilized in conjunction with various downstream tasks. For example, the system can be utilized for predicting the physical properties or structure of a protein having an amino acid sequence from protein data, predicting the binding affinity of a protein having an amino acid sequence to a specific molecule from protein data, predicting the interaction of a protein having an amino acid sequence with a specific protein from protein data, and designing a protein having superior binding affinity to a specific target. By performing downstream tasks using the protein representation learning system and method according to one embodiment of the present disclosure, significantly improved performance can be achieved compared to conventional techniques.

[0074] FIG. 1 is a flowchart illustrating a method for learning protein representation through self-supervised contrastive learning from protein amino acid sequence information according to one embodiment of the present disclosure.

[0075] FIG. 2 is a flowchart illustrating a method for learning protein expression according to one embodiment of the present disclosure.

[0076] FIG. 3 is a flowchart illustrating a system for training one or more neural networks to generate protein representations from protein amino acid sequence information and protein structure information according to one embodiment of the present disclosure.

[0077] FIG. 4 is a flowchart illustrating a method for contrastively learning protein expression from protein data according to one embodiment of the present disclosure.

[0078] FIG. 5 is a diagram showing a protein expression in which the interaction partners are expressed so that the distance between similar proteins is close to each other, in a form in which the proteins and interaction partners are combined, according to one embodiment of the present disclosure.

[0079] FIG. 6 is a diagram showing protein expressions in which interaction partners are expressed so that similar proteins are close to each other in units of proteins, according to one embodiment of the present disclosure.

[0080] FIG. 7 is a block diagram illustrating a device for performing protein representation learning from protein amino acid sequence information according to one embodiment of the present disclosure.

[0081] To clarify the technical idea of ​​the present disclosure, embodiments of the present disclosure will be described in detail with reference to the attached drawings. In describing the present disclosure, if a detailed description of a related known function or component is determined to unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. In the drawings, components having substantially the same function or configuration are given the same reference numbers and symbols as possible even if they are shown in different drawings. For convenience of explanation, devices and methods are described together when necessary. Each operation of the present disclosure does not necessarily have to be performed in the order described and may be performed in parallel, selectively, or individually.

[0082] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, the applicant may arbitrarily select terms, and in such cases, their meanings will be described in detail in the description of the relevant embodiments. Therefore, the terms used in this specification should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0083] Throughout this disclosure, singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms such as "comprise" or "have" should be understood to indicate the presence of a feature, number, step, operation, component, part, or combination thereof, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. In other words, when it is said throughout this disclosure that a part "comprises" a certain component, unless specifically stated otherwise, this does not mean that other components may be included, but rather that other components may be excluded.

[0084] Expressions such as "at least one" modify the entire list of elements, not individual elements of the list. For example, "at least one of A, B, and C" and "at least one of A, B, or C" refer to only A, only B, only C, both A and B, both B and C, both A and C, all of A, B, and C, or any combination thereof.

[0085] In addition, terms such as “...unit”, “...module”, etc. described in the present disclosure mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

[0086] Throughout this disclosure, when a part is said to be "connected" to another part, this includes not only cases where the parts are "directly connected," but also cases where the parts are "electrically connected" with other elements intervening. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather includes other components, unless otherwise specifically stated.

[0087] The expression “configured to” as used throughout this disclosure can be used interchangeably with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” does not necessarily mean something that is “specifically designed to” in terms of hardware. Instead, in some contexts, the expression “a system configured to” can mean that the system, together with other devices or components, is “capable of.” For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor (e.g., an embedded processor) for performing those operations, or a generic-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in memory.

[0088] The artificial intelligence-related functions according to the present disclosure are operated via a processor and memory. The processor may be comprised of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a Digital Signal Processor (DSP), a graphics-only processor such as a GPU or a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0089] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the predefined operation rules or artificial intelligence models are created by being learned by a learning algorithm using learning data, thereby achieving a desired purpose. This learning may be performed in the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Representative examples of the learning algorithms of the artificial intelligence model include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, self-supervised learning, or reinforcement learning.

[0090] Throughout the present disclosure, devices may include, but are not limited to, servers, smartphones, tablet PCs, PCs, TVs, smart TVs, mobile phones, personal digital assistants (PDAs), speakers, laptops, media players, microservers, e-book object recognition devices, digital broadcasting object recognition devices, kiosks, MP3 players, digital cameras, robot vacuum cleaners, home appliances, other mobile or non-mobile computing devices, watches, glasses, hair bands, and rings having communication and data processing capabilities.

[0091] The present disclosure relates to a protein representation learning system and method. Representation learning is a technology that learns to automatically generate representations from raw data. By learning protein representations, an AI model can be trained to perform downstream tasks, such as predicting a protein structure from its amino acid sequence or predicting the amino acid sequence of a protein with desired characteristics. Therefore, the representations learned by an AI model significantly impact its performance. If an AI model learns meaningful representations, it can generalize well to new downstream tasks even with a small amount of data. Therefore, generating meaningful representations can significantly improve the performance of the AI ​​model.

[0092] A protein representation learning method according to one embodiment of the present disclosure can be performed using a contrastive learning method. Contrastive learning is a type of representation learning technique that learns to generate representations so that similar data are closer together and dissimilar data are further apart in a representation space. The protein representation learning system and method according to one embodiment of the present disclosure can use a self-supervised contrastive learning method. Self-supervised contrastive learning is a highly economical and efficient method that learns vector-value representations of training data without relying on labels or annotations, and does not require data labeling.

[0093] A protein expression learning system and method according to one embodiment of the present disclosure can utilize supervised contrastive learning. Supervised contrastive learning utilizes the labels of each training data, rather than the training data itself, to determine whether the training data are similar to an anchor. Specifically, the model is trained to classify anchor labels as positive pairs if they are identical, and define them as negative pairs if they are different. A protein expression learning system and method according to one embodiment of the present disclosure can utilize labeled protein amino acid sequence data of protein interaction partners as protein data.

[0094] FIG. 1 is a flowchart illustrating a method for learning protein representation through self-supervised contrastive learning from protein amino acid sequence information according to one embodiment of the present disclosure.

[0095] Referring to FIG. 1, a device including a protein expression learning system performs protein expression learning by a method including a step of generating a protein amino acid sequence decomposed from protein amino acid sequence information that is a learning target. By sequentially performing the protein amino acid sequence decomposition step (101), encoding step (102), projection step (103), and contrast learning step (104), the device can learn to generate an accurate protein expression.

[0096] Decomposition steps of protein amino acid sequence (101)

[0097] A device including the protein expression learning system of the present disclosure obtains a plurality of protein amino acid sequences by dividing the protein amino acid sequence that is a learning target in the decomposition step.

[0098] There are no particular restrictions on the method of decomposing the protein amino acid sequence information that is the learning target, and any method that can generate an expression by appropriately reflecting the semantic composition of the protein amino acid sequence that is the learning target can be applied without restriction.

[0099] Specifically, the number of decomposed protein amino acid sequences generated by dividing the protein amino acid sequence information that is the learning target is not particularly limited as long as there are two or more. In one embodiment, the protein amino acid sequence information that is the learning target may be divided into two to obtain two decomposed protein amino acid sequences.

[0100] Furthermore, the method for dividing the protein amino acid sequence information, which is the learning target, is not particularly limited as long as a protein representation can be generated from the divided protein amino acid sequence. In one embodiment, the amino acid residues of the decomposed protein amino acid sequences may be distributed so that they are identical. If the number of amino acid residues cannot be distributed equally, the decomposed protein amino acid sequences may be divided so that the difference in the number of amino acid residues between the decomposed protein amino acid sequences is 1 or less.

[0101] Encoding step (102)

[0102] A device including the protein representation learning system of the present disclosure can use an encoder neural network to generate representations from the decomposed protein amino acid sequence data. The encoder neural network receives an input sequence and converts it into a representation, encoding a given sentence into a representation in the form of a fixed-length vector. Any encoder neural network capable of encoding protein amino acid tokens can be used without particular limitations. In one embodiment, the encoder neural network uses a transformer encoder model.

[0103] Integration stage (103)

[0104] A device including the protein expression learning system of the present disclosure integrates a plurality of intermediate expressions generated from a plurality of protein amino acid sequences in an integration step to create a single expression.

[0105] The integration step can be performed using any method that can integrate the representations generated through the encoder neural network without limitation, and it is also possible to utilize a combination of multiple methods. Methods that utilize the representations generated through the encoder neural network include methods such as summation, averaging, and concatenation, and the contrastive learning of the present disclosure can be performed by selecting one of the methods listed above.

[0106] Projection stage (104)

[0107] A device including the protein expression learning system of the present disclosure generates a projected representation from an intermediate representation generated in the encoding step or an integrated intermediate representation generated in the integration step during the projection step. The device including the protein expression learning system of the present disclosure preferably generates the projected representation through a linear projection algorithm.

[0108] Contrastive learning stage (105)

[0109] One or more neural networks generating protein representations of a device including the protein representation learning system of the present disclosure are trained to generate protein representations reflecting the semantic compositionality of protein amino acid sequence information through contrastive learning.

[0110] The contrastive learning method of the neural network of the present disclosure is preferably self-supervised contrastive learning. Self-supervised learning, here, refers to a method of training a neural network from unlabeled data. This method creates tasks that learn from the data itself, without requiring labels or annotations. This learning method enables the neural network to understand and learn from the structure and patterns inherent in the data.

[0111] A device including a protein expression learning system of the present disclosure can perform contrastive learning without a label generation process for protein amino acid sequences by performing self-supervised contrastive learning as described above, and has the advantage of being able to dramatically improve learning efficiency by not incurring the cost burden that may arise in the label generation process for protein amino acid sequences.

[0112] A device including the protein expression learning system of the present disclosure receives projections from the contrastive learning stage and evaluates a contrastive loss function. The algorithm for evaluating the contrastive loss function is not particularly limited, but may be any algorithm used for contrastive learning in conventional neural networks. For example, the algorithm used in the Simple Framework for Contrastive Learning of Visual Representations (SimCLR) may be used.

[0113] A device including a protein representation learning system of the present disclosure can improve one or more values ​​or parameters of one or more neural networks among an encoder neural network and a projection neural network based on the contrast loss function, and through this improvement step, the device including a protein representation learning system of the present disclosure can generate an accurate protein representation that reflects the semantic composition of a protein amino acid sequence.

[0114] FIG. 2 is a flowchart illustrating a learning system of one or more neural networks for generating protein representations according to one embodiment of the present disclosure.

[0115] Referring to FIG. 2, a device including a protein expression learning system of the present disclosure performs contrastive learning through the steps of acquiring learning target data, generating a representation from the learning target data, generating a representation from decomposed protein amino acid sequence learning data, and evaluating a contrastive loss function from the representations thus generated.

[0116] The protein expression learning system of the present disclosure may utilize amino acid sequence data of a protein as the learning target data. Protein amino acid sequence data refers to information in which alphabetic characters representing a total of 20 amino acid residues constituting a protein are listed in the order in which the amino acid residues are combined. The type of protein that is the learning target of the protein expression learning system of the present disclosure is not particularly limited, but amino acid sequence data of an antibody or a fragment thereof may be used, and amino acid sequence data of a complementarity determining region of an antibody or a fragment thereof may be used.

[0117] The step of generating a representation from the learning target data includes the step of inputting the learning target protein sequence data (201) into an encoder neural network (211) to obtain a first intermediate representation (212), and inputting this into a projection neural network (213) to obtain a first projected representation (214).

[0118] The step of generating an expression from the decomposed protein amino acid sequence learning data (221) consists of the following steps.

[0119] A device including a protein expression learning system of the present disclosure receives protein sequence data (201) to be learned, and then divides the protein sequence data into a plurality of sequences (101) to generate a plurality of decomposed protein amino acid sequences (221). Here, the number of protein amino acid sequences generated by division is not particularly limited, but one protein amino acid sequence may be divided into two protein amino acid sequences. The division method is not particularly limited, but the number of tokens between the plurality of decomposed protein amino acid sequences may be divided equally, and when the number of tokens of the plurality of decomposed protein amino acid sequences cannot be divided equally, the division may be performed so that the difference in the number of tokens between the plurality of decomposed protein amino acid sequences becomes 1 or less.

[0120] Afterwards, multiple decomposed protein amino acid sequences (221) are input into an encoder neural network (211) to obtain multiple second intermediate representations (222).

[0121] Next, the plurality of second intermediate representations (222) generated through the encoder neural network (221) are integrated (103) to generate an integrated second intermediate representation (223). Here, integration can be performed without limitation by utilizing any method capable of integrating the representations generated through the encoder neural network, and integration can also be performed by combining multiple methods. Methods using the representations generated through the encoder neural network include methods such as summation, averaging, and concatenation, and in the step of generating an integrated second intermediate representation by integrating one or more of the plurality of second intermediate representations, the integration can be performed by one method selected from among summation, averaging, and concatenation. Here, the summation is a method of obtaining a unified second intermediate expression (223) by adding a plurality of second intermediate expressions (222), the averaging is a method of obtaining a unified second intermediate expression (223) by adding a plurality of second intermediate expressions (222) and then dividing by the number of the plurality of second intermediate expressions (222), and the concatenation is a method of making the odd-numbered dimension value of the unified second intermediate expression (223) the odd-numbered dimension value of one of the plurality of second intermediate expressions (222), and making the even-numbered dimension value of the unified second intermediate expression (223) the even-numbered dimension value of another of the plurality of second intermediate expressions (222).

[0122] Afterwards, the integrated second intermediate representation (223) is input into the projection neural network (213) to generate a second projected representation (224).

[0123] A contrastive loss function is evaluated from the expression (214) generated from the above learning target data and the expression (224) generated from the decomposed protein amino acid sequence learning data. The algorithm for evaluating the contrastive loss function is not particularly limited and any algorithm used for contrastive learning of a conventional neural network can be used. In one embodiment, an algorithm used in a simple framework for contrastive learning of visual representations (SimCLR) is used.

[0124] A device including a protein representation learning system of the present disclosure can improve one or more values ​​or parameters of one or more neural networks among an encoder neural network and a projection neural network based on the contrast loss function, and through this improvement step, the device including a protein representation learning system of the present disclosure can generate an accurate protein representation that reflects the semantic composition of a protein amino acid sequence.

[0125] FIG. 3 is a flowchart of a method for predicting protein properties or structures by applying a protein expression learning neural network according to one embodiment of the present disclosure to downstream tasks.

[0126] FIG. 3 is an example of how the protein expression learning neural network of the present disclosure can be utilized, and the protein expression learning neural network of the present disclosure can be included in any downstream task in which a pre-trained protein expression model is utilized. For example, it can be utilized in tasks such as a task of predicting the physical properties or structure of a protein having an amino acid sequence from protein amino acid sequence information, a task of predicting the binding affinity of a protein having an amino acid sequence to a specific molecule from protein amino acid sequence information, and a task of predicting the interaction of a protein having an amino acid sequence to a specific protein from protein amino acid sequence information.

[0127] Since the protein expression learning neural network according to one embodiment of the present disclosure learns by reflecting semantic compositionality, downstream tasks including the protein expression learning neural network can provide results with improved prediction reliability compared to conventional techniques.

[0128] According to one embodiment of the present disclosure, a large-scale protein language model can be used to predict the properties of a protein from protein amino acid sequence information (201), and ESM-2 (Evolutionary Scale Modeling-2) can be used as the large-scale protein language model.

[0129] According to one embodiment of the present disclosure, the performance or structure of a protein can be predicted by utilizing the large-scale protein language model from a protein sequence that has been decomposed into protein amino acid sequence information (201).

[0130] According to one embodiment of the present disclosure, a contrastive loss function can be evaluated by inputting an expression derived from protein amino acid sequence information (201) and a large-scale protein language model (311) from a decomposed protein amino acid sequence (201) into a contrastive loss function evaluation unit, and the contrastive loss function can be evaluated using the SimCLR contrastive loss function.

[0131] According to one embodiment of the present disclosure, protein structural information (321) can be additionally utilized downstream.

[0132] According to one embodiment of the present disclosure, a predicted protein amino acid sequence can be obtained from protein structure information data (321) using a model (322) such as ESM-IF (Evolutionary Scale Modeling-Inverse Folding). Thereafter, a protein representation (324) can be generated from the protein amino acid sequence through a protein three-dimensional structure encoder neural network (324).

[0133] A contrast loss function can be obtained by inputting a contrast loss function from a protein expression (324) obtained from the protein structure information and a protein expression (214) obtained from the protein amino acid sequence information into a contrast loss function evaluation unit, and in one embodiment, the contrast loss function is a mean squared error (MSE) contrast loss function.

[0134] One or more values ​​or parameters of one or more protein expression learning neural networks can be improved based on the above contrast loss function, and through this improvement step, the accuracy of a device including the protein expression learning system of the present disclosure can be improved.

[0135] FIG. 4 is a flowchart illustrating a method for performing protein representation learning through contrastive learning from protein data according to one embodiment of the present disclosure. Referring to FIG. 4 , a device including a protein representation learning system can acquire protein data (401) as a learning target, generate protein representations (402), and perform contrastive learning (403).

[0136] Protein data acquisition step (401)

[0137] A device including the protein expression learning system of the present disclosure acquires protein data as a learning target. The protein data may include protein amino acid sequences labeled with interaction partners.

[0138] An interaction partner refers to a substance with which a protein interacts and binds. Protein interaction refers to a phenomenon in which a protein and an interaction partner come into physical contact with high specificity through interactions involving electrostatic forces, hydrogen bonding, and hydrophobic effects. The interaction partner according to the present disclosure may be a protein, but it does not have to be a protein, and may also be a low-molecular-weight organic compound, as long as it is a substance to which a protein specifically binds.

[0139] A protein according to the present disclosure may have one or more interaction partners, and in the case of a protein that binds to itself, such as a dimer, the protein and the interaction partner may be the same.

[0140] Any protein according to the present disclosure may be used without limitation. For example, it may be an antibody, in which case the interaction partner of the protein is a substance that interacts with the antibody, typically an antigen of the antibody.

[0141] The protein interaction data acquired by a device including the protein expression learning system of the present disclosure may be acquired from a protein database. The protein database that can be utilized by a device including the protein expression learning system of the present disclosure is not particularly limited, but may include, but is not limited to, STRING, BioGRID, DrugBank, BindingDB, and the like.

[0142] Protein expression generation step (402)

[0143] According to one embodiment of the present disclosure, a protein representation is generated from the acquired protein data, and can be generated from the protein data using one or more encoder neural networks. In this case, a protein representation can be generated not only using a single encoder neural network, but also by combining multiple encoder neural networks.

[0144] An encoder neural network according to one embodiment of the present disclosure can generate protein representations by comparing patterns of protein interaction partners. Any encoder neural network capable of generating protein representations can be used without limitation, and a pre-trained protein language model (e.g., ESM2) can also be used.

[0145] FIG. 5 is a diagram illustrating protein representations generated by an encoding neural network trained according to one embodiment of the present disclosure. FIG. 5 illustrates representations of an anchor protein (501) and proteins (502 and 503) classified as positive pairs, and proteins (504 and 505) classified as negative pairs with the anchor protein. In FIG. 5, a protein (502) that is identical to the anchor protein (501) is classified as a positive pair because its interaction partner (511) is identical, and a protein (503) that is different from the anchor protein (501) and has the same interaction partner is also classified as a positive pair. On the other hand, proteins (504 and 505) that are different from the anchor protein (501) and have different interaction partners (512) are classified as a negative pair, and if the interaction partners (512) of the proteins (504 and 505) classified as negative pairs are identical, the proteins generate representations so that they are close to each other.

[0146] FIG. 6 is a diagram illustrating protein representations generated by an encoding neural network trained according to one embodiment of the present disclosure. FIG. 6 illustrates protein representations generated by an encoding neural network trained according to one embodiment of the present disclosure from the amino acid sequences of four proteins (601 to 604) having different protein amino acid sequences. In FIG. 6 , protein 601 interacts with proteins 601 to 604, and protein 602 interacts with proteins 601, 603, and 604. Protein 603 interacts with proteins 601 and 602, and protein 604 also interacts with proteins 601 and 602. Here, protein 602 does not interact with the same protein, protein 603 does not interact with proteins 603 and 604, and protein 604 does not interact with proteins 603 and 604. In Figure 6, protein 601 and protein 602 are generated close to each other because their interaction partners are similar in that they interact with proteins 601, 603, and 604, and proteins 603 and 604 are generated close to each other because their interaction partners are identical in that they both interact with proteins 601 and 602. On the other hand, protein 602 interacts with proteins 603 and 604, but proteins 603 and 604 do not interact with proteins 603 and 604, so their patterns of interaction partners are different, and thus the expressions of protein 602 and protein 603 or 604 are generated far apart.

[0147] Contrastive learning stage (403)

[0148] As described above, according to one embodiment of the present disclosure, an encoding neural network can be trained to generate representations that bring together similar protein representations and that bring together dissimilar protein representations. Here, the distance between the generated representations can be measured to determine similarity. Representations with a distance below a certain distance can be considered similar, and those with a distance greater than a certain distance can be considered dissimilar.

[0149] According to one embodiment of the present disclosure, the distance between representations in contrastive learning can be calculated using various distance functions. Distance functions that can be used according to one embodiment of the present disclosure include, but are not particularly limited to, the Euclidean distance function, the Manhattan distance, the Chebychev distance, the Cosine distance, the Minkowski distance, the Levenshtein distance, the Mahalanobis distance, the Jaccard distance, and the Hamming distance.

[0150] According to one embodiment of the present disclosure, the encoding neural network can obtain a contrastive loss based on the average of the determined distances, and can be trained based on the contrastive loss thus obtained. Here, the training can be performed by modifying the values ​​of parameters of the encoding neural network based on the contrastive loss, and according to one embodiment of the present disclosure, the values ​​of one or more parameters of one or more encoder neural networks can also be modified based on the contrastive loss.

[0151] According to one embodiment of the present disclosure, an encoding neural network can be trained to generate representations based on contrastive loss that bring together similar protein representations and that bring together dissimilar protein representations. For example, training can be performed to generate representations that bring together protein representations in a positive pair relationship and that bring together protein representations in a negative pair relationship. In other words, the encoding neural network can be trained to reduce contrastive loss as the similarity of positive pairs increases, and to reduce contrastive loss as the similarity of negative pairs decreases, thereby outputting a result that minimizes contrastive loss.

[0152] An encoder neural network according to one embodiment of the present disclosure is trained to generate protein representations that reflect information about protein interaction partners through the above-described contrastive learning. Conventional protein representation learning methods and systems train to generate protein representations in which the distance between a protein and its interaction partner is close. However, since proteins and their interaction partners often have different structures, it is difficult to determine whether proteins with similar representations have similar structures or interact complementarily. When such protein representations are used for tasks such as protein structure prediction or protein drug design, there is a problem of poor performance. By training an artificial intelligence neural network according to a protein representation learning method and system according to one embodiment of the present disclosure, protein representations generated in which the distance between a protein and its interaction partner is far can be obtained, thereby resolving the problems of the conventional techniques.

[0153] According to one embodiment of the present disclosure, the protein representations obtained by training an artificial intelligence neural network using a protein representation learning method and system can be utilized in downstream tasks. The tasks for which the protein representations can be utilized are not particularly limited, but can include predicting protein properties, structures, and binding affinity for specific molecules, as well as designing amino acid sequences for proteins with desired properties. By utilizing protein representations that reflect interaction partner information in these tasks, the performance of these tasks can be significantly improved.

[0154] FIG. 7 is a block diagram of a protein expression learning device according to one embodiment of the present disclosure.

[0155] Referring to FIG. 7, the protein expression learning device (700) may include a transceiver (710), a memory (720), a database (730), and a processor (740). However, not all of the components illustrated in FIG. 7 are essential components of the protein expression learning device (700). The protein expression learning device (700) may be implemented with more components than the components illustrated in FIG. 7, or may be implemented with fewer components than the components illustrated in FIG. 7. In addition, the transceiver (710), the memory (720), and the processor (740) may be implemented in the form of a single chip.

[0156] In one embodiment, the transceiver (710) can communicate with a terminal or other electronic device connected wired or wirelessly to the protein expression learning device (700). For example, the transceiver (710) can obtain protein amino acid sequence information, protein structure information, interaction data, or protein expressions generated using an artificial neural network from the other electronic device.

[0157] The memory (720) may be installed and stored with various types of data, such as programs and files, such as applications. The processor (740) may access and use data stored in the memory (720), or store new data in the memory (720). In addition, the memory (720) may store one or more instructions. The processor (740) may execute one or more instructions stored in the memory.

[0158] The processor (740) controls the overall operation of the protein expression learning device (700) and may include at least one processor, such as a CPU or a GPU. The processor (740) may control other components included in the protein expression learning device (700) to perform operations for operating the protein expression learning device (700). For example, the processor (740) may obtain protein amino acid sequence information, obtain protein expressions using the neural network, derive a contrastive loss function from the protein expressions, and improve one or more values ​​or parameters of one or more neural networks among the encoder neural network and the projection neural network based on the contrastive loss function.

[0159] The database (730) can store various learning data for training a learning model. Furthermore, the database (730) can store protein amino acid sequence information, protein structure information, simulation result information, and the like. In various embodiments, the database (730) can also store output data generated by the learning model. While FIG. 7 illustrates the protein expression learning device (700) as including the database (730), the database (730) may be provided externally to the device. In this case, the database (730) can be connected to the protein expression learning device (700) via wired or wireless connections.

[0160] Additionally, the learning model may be implemented outside the protein expression learning device (700) (e.g., cloud-based) or may be included within the protein expression learning device (700).

[0161] An embodiment of the present disclosure may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically contains computer-readable instructions, data structures, or program modules, and includes any information delivery media.

[0162] The above description of the present disclosure is provided for illustrative purposes only, and those skilled in the art will readily appreciate that the present disclosure can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0163] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

In a system comprising at least one processor and a memory storing one or more instructions, The at least one processor learns protein representation using protein amino acid sequences by executing the one or more instructions stored in the memory, The above learning is, A step of obtaining information on one or more protein amino acid sequences, A step of generating a first intermediate representation from the protein amino acid sequence information using an encoder neural network, and generating a first projected representation from the first intermediate representation using a projection neural network; A step of generating a plurality of decomposed protein amino acid sequence information by dividing the protein amino acid sequence information, generating a plurality of second intermediate representations from the plurality of decomposed protein amino acid sequence information using the encoder neural network, and then generating an integrated second intermediate representation by integrating at least one of the plurality of second intermediate representations, and then generating a second projected representation using the projection neural network. A step of evaluating a contrast loss function from the first projected representation and the second projected representation, and A protein expression learning system comprising a step of improving one or more values ​​or parameters of one or more neural networks among the encoder neural network and the projection neural network based on the contrast loss function. A protein expression learning system, wherein in the step of generating an integrated second intermediate expression by integrating one or more of the plurality of second intermediate expressions, the integration is performed in one method selected from among summation, averaging, and concatenation. A protein expression learning system in claim 1, wherein the step of generating decomposed protein amino acid sequence information by dividing the protein amino acid sequence information is to divide the protein amino acid sequence information into two decomposed protein amino acid sequence information. A protein expression learning system in claim 1, wherein the encoder neural network is a transformer encoder model. A protein expression learning system in claim 1, wherein the projection neural network is a linear projection model. A protein expression learning system in claim 1, wherein the protein amino acid sequence information is amino acid sequence information of an antibody or a fragment thereof. A protein expression learning system in claim 6, wherein the protein amino acid sequence information is amino acid sequence information of a complementarity determining region of an antibody or a fragment thereof. A protein expression learning method performed by at least one processor, The method comprises a step of learning protein expression using a protein amino acid sequence, The above learning is, A step of obtaining information on one or more protein amino acid sequences, A step of generating a first intermediate representation from the protein amino acid sequence information using an encoder neural network, and generating a first projected representation from the first intermediate representation using a projection neural network; A step of generating a plurality of decomposed protein amino acid sequence information by decomposing the protein amino acid sequence information, generating a plurality of second intermediate representations from the plurality of decomposed protein amino acid sequence information using the encoder neural network, and then generating an integrated second intermediate representation by integrating at least one of the plurality of second intermediate representations, and then generating a second projected representation through the projection neural network. A step of evaluating a contrast loss function from the first projected representation and the second projected representation, and A protein representation learning method comprising a step of improving one or more values ​​or parameters of one or more neural networks among the encoder neural network and the projection neural network based on the contrast loss function. A protein representation learning method in claim 8, wherein in the step of generating an integrated second intermediate representation by integrating one or more of the plurality of second intermediate representations, the integration is performed in one method selected from among summation, averaging, and concatenation. A protein representation learning method in claim 8, wherein the step of generating decomposed protein amino acid sequence information by dividing the protein amino acid sequence information is to divide the protein amino acid sequence information into two decomposed protein amino acid sequence information. In claim 8, the encoder neural network is a protein expression learning system that is a transformer encoder model. In claim 8, the projection neural network is a protein expression learning system that is a linear projection model. A protein expression learning method in claim 8, wherein the protein amino acid sequence information is amino acid sequence information of an antibody or a fragment thereof. A protein expression learning method in claim 13, wherein the protein amino acid sequence information is amino acid sequence information of a complementarity determining region of an antibody or a fragment thereof. A program stored on a computer-readable recording medium that causes a computer to execute any one of the methods of Articles 8 to 14.

Images