Hierarchical contrastive learning system and method for predicting protein function

Abstract

The present invention relates to a protein function prediction system comprising: a memory storing one or more instructions; and at least one processor that executes the one or more instructions stored in the memory, wherein steps performed by the one or more instructions include the steps of: obtaining protein amino acid sequence information; and predicting, by using at least one artificial intelligence model trained through hierarchical contrastive learning, an EC number of a protein having the protein amino acid sequence information, wherein the hierarchical contrastive learning involves training using hierarchical multi-label contrastive loss.

Description

Hierarchical contrastive learning system and method for protein function prediction

[0001] The present disclosure relates to a system and method for predicting the EC number of a protein having an amino acid sequence by learning protein amino acid sequence information and the EC number assigned to the protein enzyme. More specifically, the present disclosure relates to a system and method for predicting the EC number of a protein using a model learned through hierarchical contrastive learning.

[0002] Enzymes are specialized proteins that act as biological catalysts, catalyzing biochemical reactions essential to life. Enzymes are widely used in diverse industries, including the food, pharmaceutical, and energy industries, and active research is underway to discover or improve improved enzymes.

[0003] Enzymes are classified according to the Enzyme Commission (EC) numbering system. The EC numbering system is a hierarchical classification system that categorizes enzymes based on the reaction they catalyze, and consists of four levels that describe enzyme function. Predicting the EC number of an enzyme protein is used to identify and classify its catalytic activity, a crucial task that forms the basis for new drug development and other protein- or enzyme-related projects.

[0004] Although over 200 million protein sequences have been identified, experimentally characterizing each and every one of them is impractical. Only a very small fraction of these proteins have been experimentally identified as enzymes with specific EC numbers, necessitating computational methods. While previous attempts have utilized computational methods to predict EC numbers from protein sequence information, these approaches have limitations, such as causing severe class imbalance and failing to reflect the inherent hierarchical structure of the EC number system.

[0005] The present disclosure seeks to provide a system or method for predicting EC numbers with improved performance compared to conventional systems.

[0006] Through this disclosure, it is intended to provide a protein function prediction system or method capable of predicting the function of a protein for which an EC number does not exist in a training set.

[0007] A protein function prediction system of one embodiment of the present disclosure includes a memory storing one or more commands, and at least one processor executing the one or more commands stored in the memory, wherein the operations performed by the one or more commands include a step of obtaining protein amino acid sequence information, and a step of predicting an EC number of a protein having the protein amino acid sequence information using at least one artificial intelligence model learned by hierarchical contrastive learning, wherein the hierarchical contrastive learning may be characterized in that learning is performed using a hierarchical multi-label contrastive loss.

[0008] In one embodiment, the hierarchical contrastive learning may be characterized by performing hierarchical multi-label contrastive learning for one or more EC number hierarchical levels.

[0009] In one embodiment, the hierarchical contrastive learning is learned using a training data set including training data without information for one or more EC number hierarchy levels, and the learned model may be characterized in that it predicts the EC number of a level one level higher than the EC number hierarchy level without information.

[0010] In one embodiment, the hierarchical contrastive learning

[0011] (i) When calculating the loss at the lowest level, all dimension representations are used, and as you go up the level, some dimensions of the representation used at the lower level are used to calculate the loss, or

[0012] (ii) When calculating the loss for each level, only some of the dimension representations are used, and some of the dimensions used for each level are used in calculating the loss for other levels, or

[0013] (iii) Use them separately so that the dimensions used for each level do not overlap, or

[0014] (iv) Set the number of learnable coefficient vectors of the same size as the expression vector to the same number as the number of levels, and use the product of the expression and the coefficient vector of each level when calculating the loss, or

[0015] (v) Perform it by placing as many multilayer perceptrons as the number of levels, or

[0016] It may include steps performed through the above combination.

[0017] In one embodiment, the hierarchical contrastive learning can obtain a feature vector of a protein amino acid sequence using a trained protein language model.

[0018] In one embodiment, the hierarchical contrastive learning can be performed by converting a feature vector of an amino acid sequence of the protein into a representation vector using a multilayer perceptron having three hidden layers.

[0019] In one embodiment, the method comprises the step of making an inference based on a distance between an EC number embedding and a query protein embedding, wherein the distance may be a Euclidean distance.

[0020] One embodiment of the present disclosure provides a method for predicting a protein function, which is performed by at least one processor, the method including a step of predicting a protein function using a protein amino acid sequence, wherein the prediction includes a step of obtaining protein amino acid sequence information, and a step of predicting an EC number of a protein having the protein amino acid sequence information using at least one artificial intelligence model learned by hierarchical contrastive learning, wherein the hierarchical contrastive learning may be characterized in that learning is performed using a hierarchical multi-label contrastive loss.

[0021] In one embodiment, the hierarchical contrastive learning may be characterized by performing hierarchical multi-label contrastive learning for one or more EC number hierarchical levels.

[0022] In one embodiment, the hierarchical contrastive learning is learned using a training data set including training data without information for one or more EC number hierarchy levels, and the learned model may be characterized in that it predicts the EC number of a level one level higher than the EC number hierarchy level without information.

[0023] In one embodiment, the hierarchical contrastive learning

[0024] (i) When calculating the loss at the lowest level, all dimension representations are used, and as you go up the level, some dimensions of the representation used at the lower level are used to calculate the loss, or

[0025] (ii) When calculating the loss for each level, only some of the dimension representations are used, and some of the dimensions used for each level are used in calculating the loss for other levels, or

[0026] (iii) Use them separately so that the dimensions used for each level do not overlap, or

[0027] (iv) Set the number of learnable coefficient vectors of the same size as the expression vector to the same number as the number of levels, and use the product of the expression and the coefficient vector of each level when calculating the loss, or

[0028] (v) Perform it by placing as many multilayer perceptrons as the number of levels, or

[0029] It can be performed including steps performed through the above combination.

[0030] In one embodiment, the hierarchical contrastive learning can obtain a feature vector of a protein amino acid sequence using a trained protein language model.

[0031] In one embodiment, the hierarchical contrastive learning can be performed by converting a feature vector of an amino acid sequence of the protein into a representation vector using a multilayer perceptron having three hidden layers.

[0032] In one embodiment, the method comprises the step of making an inference based on a distance between an EC number embedding and a query protein embedding, wherein the distance may be a Euclidean distance.

[0033] One embodiment of the present disclosure can provide a program stored on a computer-readable recording medium to execute the above method on a computer.

[0034] One embodiment of the present disclosure provides a computerized method for predicting protein function,

[0035] The computerized method for predicting the protein function includes the steps of inputting protein amino acid sequence information into an artificial intelligence model executed by a processor, and the step of predicting the EC number of a protein having the protein amino acid sequence information using the artificial intelligence model learned by hierarchical contrastive learning executed by at least one processor, wherein the hierarchical contrastive learning may be characterized in that learning is performed using a hierarchical multi-label contrastive loss.

[0036] In one embodiment, the hierarchical contrastive learning may be characterized by performing hierarchical multi-label contrastive learning for one or more EC number hierarchical levels.

[0037] In one embodiment, the hierarchical contrastive learning is learned using a training data set including training data without information for one or more EC number hierarchy levels, and the learned model may be characterized in that it predicts the EC number of a level one level higher than the EC number hierarchy level without information.

[0038] In one embodiment, the hierarchical contrastive learning

[0039] (i) When calculating the loss at the lowest level, all dimension representations are used, and as you go up the level, some dimensions of the representation used at the lower level are used to calculate the loss, or

[0040] (ii) When calculating the loss for each level, only some of the dimension representations are used, and some of the dimensions used for each level are used in calculating the loss for other levels, or

[0041] (iii) Use them separately so that the dimensions used for each level do not overlap, or

[0042] (iv) Set the number of learnable coefficient vectors of the same size as the expression vector to the same number as the number of levels, and use the product of the expression and the coefficient vector of each level when calculating the loss, or

[0043] (v) Perform it by placing as many multilayer perceptrons as the number of levels, or

[0044] It can be performed including steps performed through the above combination.

[0045] In one embodiment, the hierarchical contrastive learning may obtain a feature vector of a protein amino acid sequence using a trained protein language model.

[0046] According to one embodiment of the present disclosure, the EC number of a protein having an input amino acid sequence can be predicted with improved performance from protein amino acid sequence information using at least one artificial intelligence model learned by hierarchical contrastive learning.

[0047] According to one embodiment of the present disclosure, the EC number of a protein having an amino acid sequence can be predicted with improved performance compared to the prior art, not only at low levels but also at high levels.

[0048] According to one embodiment of the present disclosure, the EC number of a protein having an amino acid sequence can be predicted with improved performance by extending the hierarchical multi-label contrast of the present disclosure to more levels of the EC number hierarchy.

[0049] According to one embodiment of the present disclosure, high-level EC numbers can be predicted with high confidence, and even higher-level EC numbers can be predicted with improved performance for protein enzymes that do not have EC numbers in the training data set. The protein function prediction system and method of the present disclosure can provide highly reliable prediction results even for novel proteins.

[0050] Figure 1 illustrates the Enzyme Commission (EC) numbering system, a hierarchical system that classifies enzymes based on the chemical reactions they catalyze. Lower levels of the EC numbering system catalyze more specific reactions.

[0051] FIG. 2 is a block diagram illustrating a device for predicting the function of a protein by predicting the EC number of the protein from protein amino acid sequence information according to one embodiment of the present disclosure.

[0052] Figures 3a through 3c illustrate the performance of hierarchical multi-label contrastive models for EC number hierarchies at levels 3 and 4. The left panel shows AUPRC, the center panel shows F1 scores with varying weights for levels 3 and 4. The right panel shows UMAP visualizations of level 4 EC number embeddings, with colors based on their level 1 EC numbers.

[0053] Figure 4 shows the performance of a model trained using hierarchical multi-label contrastive learning for levels 1 to 4 of the EC number hierarchy. The vertical line in the table represents the performance when hierarchical multi-label contrastive learning is applied only to level 4.

[0054] Figure 5 shows the performance of hierarchical multi-label contrast when applied to EC numbers with no information. Each column represents the performance for level l when information about the level at level l+1 in the training set is missing.

[0055] Figures 6a and 6b show the results of comparing the performance of the model of the present disclosure with a conventional model. The left side shows performance on the New-392 dataset, the right side shows performance on the Price-149 dataset, and the vertical line shows the performance of a hierarchical multi-label contrastive model applied only to level 4.

[0056] Figure 7 shows the robustness of the prediction results.

[0057] Figure 8 presents the results of a specific case study of the function of the enzyme A0A1D8PH52 from the NEW-392 dataset. Initially misclassified as 2. 3. 3. 10 (hydroxymethylglutaryl-CoA synthase), it was correctly identified as 2. 3. 1. 9 (acetyl-CoA C-acetyltransferase) by incorporating EC number hierarchy.

[0058] To clarify the technical idea of ​​the present disclosure, embodiments of the present disclosure will be described in detail with reference to the attached drawings. In describing the present disclosure, if a detailed description of a related known function or component is determined to unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. In the drawings, components having substantially the same function or configuration are given the same reference numbers and symbols as possible even if they are shown in different drawings. For convenience of explanation, devices and methods are described together when necessary. Each operation of the present disclosure does not necessarily have to be performed in the described order and may be performed in parallel, selectively, or individually.

[0059] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, the applicant may arbitrarily select terms, and in such cases, their meanings will be described in detail in the description of the relevant embodiments. Therefore, the terms used in this specification should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0060] Throughout this disclosure, singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms such as "comprise" or "have" should be understood to indicate the presence of a feature, number, step, operation, component, part, or combination thereof, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. In other words, when it is said throughout this disclosure that a part "comprises" a certain component, unless specifically stated otherwise, this does not mean that other components may be included, but rather that other components may be excluded.

[0061] Expressions such as "at least one" modify the entire list of elements, not individual elements of the list. For example, "at least one of A, B, and C" and "at least one of A, B, or C" refer to only A, only B, only C, both A and B, both B and C, both A and C, all of A, B, and C, or any combination thereof.

[0062] In addition, terms such as “...part”, “...module”, etc. described in the present disclosure mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

[0063] Throughout this disclosure, when a part is said to be "connected" to another part, this includes not only cases where the parts are "directly connected," but also cases where the parts are "electrically connected" with other elements intervening. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather includes other components, unless otherwise specifically stated.

[0064] The expression "configured to" as used throughout this disclosure can be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" does not necessarily mean something is "specifically designed to" in hardware. Instead, in some contexts, the expression "a system configured to" can mean that the system is "capable of" in conjunction with other devices or components. For example, the phrase "a processor configured to perform A, B, and C" can mean a dedicated processor for performing the operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) that can perform the operations by executing one or more software programs stored in memory.

[0065] The artificial intelligence-related functions according to the present disclosure are operated via a processor and memory. The processor may be comprised of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a Digital Signal Processor (DSP), a graphics-only processor such as a GPU or a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0066] The protein of the present disclosure is designated by one or more amino acid sequences. Amino acids are organic compounds that contain amino and carboxyl functional groups as well as side chains (i.e., groups of atoms) specific to amino acids.

[0067] Protein folding refers to the physical process by which a sequence of amino acids folds into a three-dimensional structure. A protein's structure defines the three-dimensional arrangement of atoms in its amino acid sequence after the protein undergoes protein embedding. When the sequence is linked by peptide bonds, the amino acids can be referred to as amino acid residues.

[0068] Enzymes are biocatalysts that bind to substrates to form enzyme-substrate complexes, thereby lowering the activation energy of chemical reactions and thus increasing the rate of metabolism. In some cases, they also perform a bioprotective function that regulates the rate of reaction. Enzymes convert substrates into other molecules known as products. Like other catalysts, enzymes increase the rate of chemical reactions by lowering the activation energy. Some enzymes can accelerate the conversion of substrates to products millions of times faster.

[0069] Enzymes are known to catalyze thousands of biochemical reactions. While most enzymes are proteins, some are RNA molecules with catalytic functions. These catalytic RNA molecules are called ribozymes. Enzymes' specificity stems from their unique three-dimensional structures.

[0070] The EC number hierarchical system is a numerical classification system for classifying enzyme proteins based on the chemical reaction they catalyze. It is also known as the enzyme number or enzyme committee number. The EC number designates the reaction catalyzed by the enzyme, and different enzymes catalyzing the same reaction are assigned the same EC number. Even when completely different protein folds catalyze the same reaction through convergent evolution, the same EC number is assigned. Therefore, the EC number of a protein enzyme can be used to determine its function.

[0071] As shown in Figure 1, all EC numbers consist of the EC letter followed by four digits separated by periods. The first digit following the EC letter is called the first level, followed by levels 2 through 4. These four levels form a hierarchy, with the first level being the highest level and the fourth level being the most detailed sublevel.

[0072] The present disclosure relates to a system for predicting an EC number from an amino acid sequence of an enzyme. Specifically, the system relates to a system for predicting an EC number from an amino acid sequence of an enzyme using an artificial intelligence model.

[0073] Artificial intelligence (AI) is a field of computer engineering and information technology that studies methods to enable computers to perform human-like tasks like thinking, learning, and self-improvement. It aims to enable computers to mimic human intelligent behavior. Furthermore, AI does not exist in isolation; rather, it is closely linked, both directly and indirectly, to other fields of computer science. In particular, efforts are actively underway to incorporate AI elements into various fields of information technology and utilize them to solve problems in those fields.

[0074] Machine learning is a branch of artificial intelligence that empowers computers to learn without explicit programming. Specifically, machine learning is the study and development of systems and algorithms that learn from empirical data, make predictions, and improve their own performance. Rather than executing strictly defined, static program instructions, machine learning algorithms build specific models based on input data to derive predictions or decisions. The term "machine learning" can be used interchangeably with "machine learning."

[0075] Many machine learning algorithms have been developed to classify data. Representative examples include decision trees, Bayesian networks, support vector machines (SVMs), and artificial neural networks (ANNs).

[0076] Decision trees are an analytical method that performs classification and prediction by graphically representing decision rules in a tree-like structure. Bayesian networks are models that represent probabilistic relationships (conditional independence) between multiple variables in a graph structure. Bayesian networks are suitable for data mining through unsupervised learning.

[0077] Support vector machines (SVMs) are supervised learning models for pattern recognition and data analysis, primarily used for classification and regression analysis. Artificial neural networks (ANNs) model the operating principles and interconnected relationships of biological neurons. They are information processing systems in which numerous neurons, called nodes or processing elements, are connected in layers.

[0078] An artificial neural network (ANN) is a model used in machine learning. It is a statistical learning algorithm inspired by biological neural networks (especially the brain, the central nervous system of animals) in machine learning and cognitive science. Specifically, an ANN can refer to a general model in which artificial neurons (nodes) form a network through the connection of synapses, which change the strength of the synaptic connections through learning, thereby achieving problem-solving capabilities. The term "ANN" can be used interchangeably with the term "neural network."

[0079] An artificial neural network can include multiple layers, each of which can contain multiple neurons. Furthermore, an artificial neural network can include synapses, which connect neurons. An artificial neural network can generally be defined by three factors: the connection patterns between neurons in different layers, a learning process that updates the weights of these connections, and an activation function that generates output values ​​from a weighted sum of inputs received from the previous layer.

[0080] Artificial neural networks may include, but are not limited to, network models such as Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), Multilayer Perceptrons (MLPs), and Convolutional Neural Networks (CNNs). In this specification, the term "layer" may be used interchangeably with the term "layer."

[0081] Artificial neural networks are categorized into single-layer neural networks and multi-layer neural networks based on the number of layers. A typical single-layer neural network consists of an input layer and an output layer. A typical multi-layer neural network consists of an input layer, one or more hidden layers, and an output layer.

[0082] The input layer is the layer that receives external data, and the number of neurons in the input layer is the same as the number of input variables, and the hidden layer is located between the input layer and the output layer. It receives signals from the input layer, extracts characteristics, and transmits them to the output layer. The output layer receives signals from the hidden layer and outputs output values ​​based on the received signals. The input signals between neurons are multiplied by each connection strength (weight) and then added, and if this sum is greater than the threshold of the neuron, the neuron is activated and outputs the output value obtained through the activation function.

[0083] Meanwhile, deep neural networks, which include multiple hidden layers between the input and output layers, are representative artificial neural networks that implement deep learning, a type of machine learning technique. The term "deep learning" can be used interchangeably with "deep learning."

[0084] The workflow of machine learning consists of a series of steps: collecting data for learning and validation, modeling, and then training the model. This can include the processes of training data collection, data inspection and exploration, data preprocessing and cleaning, modeling, and training.

[0085] 1. Collect training data

[0086] The training data used to train the artificial intelligence model of this gas can be generated using data collected from multiple samples. Each of the multiple samples can contain characteristic information about the enzyme.

[0087] For example, enzyme characteristic information includes the amino acid sequence, function, structure, post-translational regulation, EC number of the enzyme, etc. of the protein, and such characteristic information can be obtained from databases provided by NCBI (National Center for Biotechnology Information) maintained by the National Institutes of Health in the United States, UniProt Knowledge Base (UniProtKB) provided by the Swiss Bioinformatics Institute, and Swiss-Prot database. However, enzyme characteristic information is not limited to this, and may include any information that can distinguish between enzymes. In addition, in some cases, multiple samples may be collected from enzymes associated with the input data to be processed by the trained model.

[0088] In this specification, at least one or more different types of training data sets may be used to train a learning model, and each training data set may further include one or more experiment-based results used as feature labels. At least a portion of the training data set may be used to train the learning model, and another portion may be used to validate the learned learning model.

[0089] 2. Checking and exploring data

[0090] Once training data for learning a learning model is collected, the collected training data can be inspected and explored for data structure, noisy data, and data cleaning methods for applying machine learning.

[0091] This data review and exploration phase is called Exploratory Data Analysis (EDA), and EDA can be defined as the process of observing and understanding the collected data from various perspectives. Before data learning, visualizations such as graphs and statistical tests are used to examine independent and dependent variables, variable types, and their data types, allowing for preliminary identification of data characteristics and inherent structural relationships. Through EDA, data distribution and values ​​can be examined to better understand the phenomena expressed by the data and identify potential problems. Furthermore, through the process of examining data from various perspectives, various patterns that might not have been detected during the problem definition phase can be discovered, allowing for modification of existing hypotheses or the development of new ones. Exploratory data analysis can broadly include the process of searching for outliers and analyzing the relationships between data attributes.

[0092] The process of detecting outliers involves determining whether data contains outliers. This process can involve sampling, statistical methods, and visualization methods. Sampling methods extract a random sample from the data to identify overall trends and anomalies in the data values. Statistical methods can utilize summary statistics such as the mean, median, and mode to determine the center of the data, or the range and variance to determine the distribution of the data. Visualization methods can utilize probability density functions, histograms, dot plots, word clouds, time series charts, and maps to determine which statistical indicators are appropriate for each attribute of the collected data. However, when using statistical indicators, it is important to note that the average reflects all data values ​​in a set, so outliers can affect the value, whereas the median uses the single value in the middle, so it can produce representative results even with outliers.

[0093] The process of analyzing the relationship between data attributes is to find combinations of attributes that have meaningful correlations with each other within the data. The relationship analysis can be performed differently depending on the combination of attributes between qualitative attributes (Categorical Variable, Qualitative) that cannot be expressed numerically but can be arbitrarily quantified and quantitative attributes (Numeric Variable, Quantitative). The qualitative-qualitative relationship (Categorical - Categorical) can be displayed by using cross-tabulations and mosaic plots to count the number of values ​​corresponding to each pair of attribute values. The quantitative-qualitative relationship (Numeric-Categorical) can be visually expressed by observing statistical values ​​(mean, median, etc.) by category or using box plots. The quantitative-quantitative relationship (Numeric-Numeric) can be analyzed for the association between two attributes using correlation coefficients. A correlation coefficient of -1 indicates a negative correlation where the two attributes change in opposite directions, 0 indicates no correlation, and 1 indicates a positive correlation where the two attributes always change in the same direction. The relationship between two attributes with a correlation coefficient can take many forms, and this can be visually represented using a scatter plot.

[0094] 3. Data preprocessing, embedding, and cleaning

[0095] Once the data has been inspected and explored, data preprocessing is performed to transform it into a format suitable for machine learning training models. Data preprocessing involves refining data and transforming it into a form understandable by the model. Data preprocessing typically includes handling missing data, removing outliers, scaling, categorical data encoding, feature selection and extraction, and data transformation. The detailed data preprocessing steps can be performed in whole or in part, and a separate machine learning model may be used for data preprocessing.

[0096] Handling missing data involves handling missing values ​​in data. Missing values ​​can be displayed as NaN (Not a Number) or blank, or they can be deleted. Filling in or deleting missing values ​​within the data improves data completeness. When filling in missing values, values ​​such as the mean, median, or mode can be used.

[0097] Outlier removal is the process of removing outliers, values ​​that deviate from the normal data pattern. Outliers can degrade model performance and should therefore be removed or replaced. Identifying outliers can be accomplished by deleting the corresponding rows or columns or replacing them with different values.

[0098] Data scaling is the process of adjusting the size of data. Through data scaling, the range of data can be adjusted, and the performance of the model or the convergence speed can be improved. Through data scaling, the characteristics of the data can be adjusted to a similar range, and generally, standardization and normalization can be applied to data scaling. Standardization is a method of converting data into a distribution with a mean of 0 and a standard deviation of 1, and is mainly converted using the mean and standard deviation, and the standardized value z can be expressed as z = (x-μ) / σ (x is the original value, μ is the mean, and σ is the standard deviation). Normalization is a method of converting the range of data to [0,1] or [-1,1], and is mainly converted using the minimum and maximum values, and the normalized value x norm Silver x norm =(xx min ) / (x max -x min ) can be expressed as (x is the original value, x min is the minimum, x max is the maximum value).

[0099] Categorical data encoding is the process of converting categorical variables, represented as strings or integers that cannot be directly input into a model, into numerical data types that can be input into the model. Typically, one-hot encoding or label encoding is used to convert categorical variables into numerical data types.

[0100] Feature selection and extraction is a process to improve model performance by selecting the most useful features for model learning or extracting new features. This process can reduce model complexity and prevent overfitting.

[0101] Data transformation is the process of transforming data to extract new information or to improve model understanding. This can include tokenizing text data or preprocessing image data. Data transformation can extract useful features from source data or transform data into an appropriate format, improving model performance.

[0102] Through data preprocessing as described above, the performance of machine learning models can be improved and stability can be secured.

[0103] Meanwhile, if the collected protein amino acid sequence-based data has not been preprocessed to suit the needs, tokenization, cleaning, and normalization can be performed to suit the intended use of the data.

[0104] For computers to understand and process text, it must be appropriately converted into numbers. Because the performance of natural language processing can vary significantly depending on how words are represented, numerous techniques have been proposed to quantify words. Currently, the most widely used method is word embedding, which vectorizes each word through artificial neural network learning. This same word embedding approach can also be applied to protein amino acid sequences.

[0105] The system of the present disclosure includes one or more artificial intelligence models that can embed protein amino acid sequence information and express it as a vector. The artificial intelligence models utilize a transformer architecture to identify evolutionary relationships and predict protein structures. Pre-trained language models can be used as the artificial intelligence models. For example, ESM-2 (Evolutionary Scale Modeling-2) can be used as the pre-trained language model.

[0106] One or more artificial intelligence models included in the system of the present disclosure can embed EC numbers and represent them as vectors. Using the artificial intelligence models, the embedding of each EC number in the training data set can be obtained as the center of the embedding of the protein associated with that EC number.

[0107] 4. Modeling, Training, and Learning

[0108] Artificial neural networks can be trained using a training data set. Here, "training" refers to the process of determining the parameters of an artificial neural network using training data to achieve objectives such as classifying input data, performing regression analysis, or performing clustering. Representative examples of artificial neural network parameters include the weights assigned to synapses and the biases applied to neurons.

[0109] An artificial neural network trained using training data can classify or cluster input data based on its patterns. Meanwhile, an artificial neural network trained using training data may be referred to herein as a learning model.

[0110] The following explains the learning methods of artificial neural networks. Learning methods of artificial neural networks can be broadly categorized into supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning.

[0111] Supervised learning is a machine learning method that infers a function from training data. Among these inferred functions, regression analysis is the process of outputting continuous values, while classification is the process of predicting and outputting the class of an input vector.

[0112] In supervised learning, an artificial neural network is trained with labels for training data. Here, the label can mean the correct answer (or result value) that the artificial neural network should infer when training data is input to the artificial neural network. In this specification, the correct answer (or result value) that the artificial neural network should infer when training data is input is called a label or labeling data. In addition, in this specification, setting a label on training data for learning of the artificial neural network is called labeling the training data. In this case, the training data and the label corresponding to the training data constitute a single training set, and can be input to the artificial neural network in the form of a training set.

[0113] Meanwhile, training data represents multiple features, and labeling the training data can mean that the features represented by the training data are labeled. In this case, the training data can represent the features of the input object in vector form. An artificial neural network can use the training data and labeled data to infer a function regarding the relationship between the training data and the labeled data. Furthermore, the parameters of the artificial neural network can be determined (optimized) by evaluating the function inferred by the artificial neural network.

[0114] Unsupervised learning is a type of machine learning in which training data is not labeled. Specifically, unsupervised learning can be a learning method that trains an artificial neural network to find and classify patterns in the training data itself, rather than the relationship between the training data and the corresponding labels. Examples of unsupervised learning include clustering and independent component analysis (ICA). In this specification, the term "clustering" may be used interchangeably with the term "clustering."

[0115] Examples of artificial neural networks that utilize unsupervised learning include generative adversarial networks (GANs) and autoencoders (AEs).

[0116] Generative adversarial networks (GANs) are a machine learning method in which two different AI components, a generator and a discriminator, compete to improve performance. In this case, the generator is a model that creates new data, capable of generating new data based on original data. The discriminator, a model that recognizes data patterns, can determine whether the input data is original or new data generated by the generator. The generator learns from data that fails to fool the discriminator, while the discriminator learns from data that the generator deceives. Accordingly, the generator can evolve to fool the discriminator as effectively as possible, while the discriminator can evolve to effectively distinguish between original data and data generated by the generator.

[0117] An autoencoder is a neural network that aims to reproduce the input itself as an output. An autoencoder comprises an input layer, at least one hidden layer, and an output layer. In this case, since the number of nodes in the hidden layer is smaller than that in the input layer, the data dimensionality is reduced, leading to compression or encoding. Furthermore, data output from the hidden layer is fed into the output layer. In this case, since the number of nodes in the output layer is larger than that in the hidden layer, the data dimensionality increases, leading to decompression or decoding.

[0118] Meanwhile, autoencoders express input data as hidden layer data by adjusting the connection strengths of neurons through learning. The hidden layer expresses information with a smaller number of neurons than the input layer. The ability to reproduce input data as output implies that the hidden layer has discovered and expressed hidden patterns in the input data.

[0119] Semi-supervised learning is a type of machine learning that utilizes both labeled and unlabeled training data. One technique for semi-supervised learning is to infer labels for unlabeled training data and then use these inferred labels to perform training. This technique can be useful in situations where labeling is expensive.

[0120] Reinforcement learning is the theory that, if an agent is given an environment in which it can determine the optimal action at any given moment, it can find the optimal path through experience without data. Reinforcement learning is primarily implemented using a Markov Decision Process (MDP). A Markov Decision Process is described as follows: first, an environment containing the information necessary for the agent to take the next action is provided; second, how the agent will act in that environment is defined; third, what rewards the agent will receive for performing well and what penalties will be imposed for performing poorly is defined; and fourth, the optimal policy is derived through repeated experience until the future reward reaches its maximum.

[0121] The structure of an artificial neural network is specified by the model configuration, activation function, loss function or cost function, learning algorithm, optimization algorithm, etc., and the hyperparameters are set in advance before learning, and the model parameters are set through learning afterwards, so that the content can be specified.

[0122] For example, factors that determine the structure of an artificial neural network may include the number of hidden layers, the number of hidden nodes included in each hidden layer, the input feature vector, and the target feature vector.

[0123] Hyperparameters include various parameters that must be initially set for learning, such as initial values ​​for model parameters. Furthermore, model parameters include various parameters to be determined through learning. For example, hyperparameters may include initial values ​​for inter-node weights, initial values ​​for inter-node biases, mini-batch size, number of learning iterations, and learning rates. Furthermore, model parameters may include inter-node weights, inter-node biases, and more.

[0124] The loss function can be used as an indicator (standard) to determine the optimal model parameters during the learning process of an artificial neural network. In an artificial neural network, learning refers to the process of manipulating model parameters to reduce the loss function, and the purpose of learning can be seen as determining the model parameters that minimize the loss function. The loss function can mainly use the mean squared error (MSE) or the cross entropy error (CEE), but the present invention is not limited thereto. The cross entropy error can be used when the correct answer label is one-hot encoded. One-hot encoding is an encoding method that sets the correct answer label value to 1 only for neurons corresponding to the correct answer, and sets the correct answer label value to 0 for neurons that are not the correct answer.

[0125] In machine learning or deep learning, learning optimization algorithms can be used to minimize the loss function. Learning optimization algorithms include gradient descent (GD), stochastic gradient descent (SGD), momentum, Nesterov Accelerate Gradient (NAG), Adagrad, AdaDelta, RMSProp, Adam, and Nadam.

[0126] Gradient descent is a technique that adjusts model parameters in a direction that reduces the loss function value by considering the gradient of the loss function at the current state. The direction of model parameter adjustment is called the step direction, and the size of the adjustment is called the step size. Here, the step size can represent the learning rate. Gradient descent obtains the gradient by partially differentiating the loss function with respect to each model parameter, and updates the model parameters by changing the learning rate in the direction of the obtained gradient.

[0127] Stochastic gradient descent is a technique that divides learning data into mini-batches and performs gradient descent on each mini-batch to increase the frequency of gradient descent.

[0128] Adagrad, AdaDelta, and RMSProp are techniques for improving optimization accuracy by adjusting the step size in SGD. In SGD, momentum and NAG are techniques for improving optimization accuracy by adjusting the step direction. Adam combines momentum and RMSProp to improve optimization accuracy by adjusting the step size and step direction. Nadam combines NAG and RMSProp to improve optimization accuracy by adjusting the step size and step direction.

[0129] The learning speed and accuracy of artificial neural networks are significantly influenced by not only the network structure and the type of learning optimization algorithm, but also hyperparameters. Therefore, to obtain a good learning model, it is crucial to not only determine an appropriate artificial neural network structure and learning algorithm, but also set appropriate hyperparameters.

[0130] Typically, hyperparameters are experimentally set to various values ​​to train the artificial neural network, and the learning results are set to the optimal values ​​that provide stable learning speed and accuracy.

[0131] The hierarchical contrastive learning of the present disclosure can be performed using a hierarchical multi-label contrastive loss as a loss function.

[0132] In one embodiment of the present disclosure, the hierarchical multi-label contrastive loss can be obtained as follows. When the hierarchical level is denoted by l∈ L, the paired loss of anchor samples and positive samples can be defined by the following equation (1).

[0133]

[0134] In the above equation (1), i represents an anchor sample, p represents a positive sample, and l represents a level. Ai is an index set of batches excluding i, τ is a temperature parameter, and f represents a feature vector.

[0135] The hierarchical multi-label contrastive loss can be obtained by using the pair loss obtained in the above equation (1) according to the following equation (2).

[0136]

[0137] In the above equation (2), λ l is the penalty weight for each level (l) of the plan, and P l refers to the set of positive samples at level l, and P l (i) denotes the set of positive samples at level l for the anchor sample indexed by i.

[0138] By leveraging all levels of the hierarchy, the hierarchical multi-label contrastive loss promotes pairs that share more ancestry closer together. This allows the model to output embedding vectors that preserve the label hierarchy within the embedding space. Furthermore, positive pairs that share a hierarchy at a certain level can become negative pairs at lower levels, thus serving as hard-negative samples, reducing the need for hard-negative mining. The penalty weights in Equation (2) can be adjusted to an optimal point that achieves the best performance.

[0139] 5. Inference using artificial intelligence models

[0140] One or more artificial intelligence models included in the system of the present disclosure can infer an EC number of a query protein from a query protein. The inference is performed based on the distance between the query protein and the EC number.

[0141] The above inference may include the steps of obtaining an embedding of each EC number from the EC numbers of the training data set, using the Euclidean distance of the EC number embedding and the query protein embedding for inference, assuming that if an enzyme belongs to a particular descendant EC number, it also belongs to the corresponding parent EC number, using the shortest distance to the descendant EC number as the distance to the parent EC number, and applying the maximum separation method to binarize the labels according to the distance.

[0142] In one embodiment of the present disclosure, the system and method for predicting protein function can be used to predict the function of an entirely new enzyme protein for which an EC number has not been previously defined. Without information on the EC number, it is impossible to predict the EC number at level 4, the lowest hierarchy level, but it is possible to predict the EC number at level 3, which is one level higher.

[0143] In one embodiment of the present disclosure, hierarchical contrastive learning can be performed using a training data set including training data having no information for one or more EC number hierarchical levels, and the learned model can predict the EC number of a level one level higher than the EC number hierarchical level having no information with high reliability.

[0144] 6. AI Model Architecture

[0145] The present disclosure proposes a contrastive learning model suitable for multi-label settings. In one embodiment of the present disclosure, a protein language model well known in the art can be used to embed protein amino acid sequences, or a pre-trained protein language model can be used. Meanwhile, a multilayer perceptron can be used to represent proteins from the embeddings.

[0146] Meanwhile, the present disclosure proposes a contrastive learning model structure considering hierarchical multi-labels to further improve the prediction performance of the model, and in one embodiment of the present disclosure, contrastive learning is

[0147] (i) When calculating the loss at the lowest level, all dimension representations are used, and as you go up the level, some dimensions of the representation used at the lower level are used to calculate the loss, or

[0148] (ii) When calculating the loss for each level, only some of the dimension representations are used, and some of the dimensions used for each level are used in calculating the loss for other levels, or

[0149] (iii) Use them separately so that the dimensions used for each level do not overlap, or

[0150] (iv) Set the number of learnable coefficient vectors of the same size as the expression vector to the same number as the number of levels, and use the product of the expression and the coefficient vector of each level when calculating the loss, or

[0151] (v) train by setting up separate multilayer perceptron layers equal to the number of levels, or

[0152] Disclosed is a model structure that performs a combination of one or more of the structures (i) to (v) described above. When the model structure is used in conjunction with a hierarchical multi-label contrastive loss to perform contrastive learning, it is expected to improve EC number prediction performance. When applied to other multi-label contrastive learning situations, it is also expected to improve performance in downstream tasks.

[0153] Mixture of Experts (MoE) architecture

[0154] In one embodiment of the present disclosure, a system and method for predicting the EC number of a protein amino acid sequence may be performed utilizing a model architecture such as MoE. Here, MoE may refer to a machine learning model architecture that combines multiple expert models to solve complex problems.

[0155] Such MoEs may include expert models, which are multiple smaller networks designed to learn different parts and / or different characteristics of a given data and perform data processing operations accordingly, and a gating network that evaluates the performance of each expert model and, based on the performance thereof, determines which expert model is most appropriate to assign a specific task to for a given data.

[0156] Thus, according to the MoE architecture, a gating network that acquires a given input data determines a probabilistic or deterministic task assignment for each expert model, and the selected expert models perform their own tasks and return the results, thereby performing data processing for a specific task.

[0157] The MoE model used in accordance with one embodiment of the present disclosure may refer to a predetermined MoE model implemented according to a common method known in the art. For example, the MoE model may include a Switch Transformer, Conditional Computation in Neural Networks, a Sparse Mixture of Experts, and / or a Megatron-LM.

[0158] Additionally, in one embodiment of the present disclosure, a MoE model based on a combination of multiple specialized models (SM) and routers (Router, Gating Network, RT) may be included, and a MoE model based on a domain-specific specialized model may be included.

[0159] By utilizing such MoE, the overall efficiency and performance of the system or method for predicting the EC number of a protein amino acid sequence of the present disclosure can be improved by concentrating computational resources by activating only specific parts, such as when dealing with complex tasks or large data sets.

[0160] 7. Computing system device

[0161] Meanwhile, embodiments of the present invention may be implemented as an application-specific integrated circuit (ASIC) manufactured to suit the special functions of a specific application field and device.

[0162] An application-specific integrated circuit is also called an application-specific semiconductor. Unlike standard semiconductors that have set specifications and can be applied to any electronic product or application as long as certain requirements are met, an application-specific semiconductor is an integrated circuit that a semiconductor manufacturer manufactures according to a specific order for a specific product or function. In other words, an application-specific semiconductor is designed and manufactured to perform only the functions required for a specific device or specific function. Depending on the design method, application-specific semiconductors are largely divided into full custom ICs, which design and manufacture the circuit from scratch according to the user's needs, and semi-custom ICs, which design and manufacture the circuit using some of the standardized design.

[0163] Application-specific semiconductors are primarily used in communications systems, high-performance computing systems, consumer electronics, automobiles, industrial automation, medical devices, military, and aerospace industries. Recently, they are being applied to AI semiconductors that perform large-scale calculations required for AI implementation with high performance and power efficiency.

[0164] Application-specific integrated circuits (ASICs) are core components of network routers, switches, and modems in communication systems, performing data packet processing, protocol conversion, and signal processing to deliver high throughput and low latency. In high-performance computing systems, ASICs are key components for high-speed and parallel processing. In consumer electronics such as digital cameras, smartphones, tablets, and game consoles, ASICs provide high-performance and low-power solutions required to perform specific functions. In the automotive industry, ASICs control various electronic systems within vehicles, and in industrial automation systems, ASICs provide solutions for high-precision control and high-performance processing.

[0165] An application-specific integrated circuit to which an embodiment of the present invention is applied includes a memory in which an individual memory interface (I / F) is implemented, and may include a plurality of functional blocks that request memory access. Each functional block may be a direct memory access (DMA) functional block, a processor, a video processor, a cache controller, a decompression block, or a data path block. The basic configuration of the application-specific integrated circuit may include a transistor that amplifies or switches an electrical signal, a logic gate that is a circuit that performs a logical function by combining transistors, a memory cell that stores data, an analog circuit that is a circuit that processes a continuous voltage or current by combining transistors, and an IP core (Intellectual Property Core) such as a microprocessor, DSP, or graphic core that is pre-designed to perform a specific function.

[0166] The ASIC may also include a separate memory I / F interfacing with individual memories and an embedded memory I / F interfacing with embedded memories. The separate memory I / F is connected to each functional block, receives memory access signals (e.g., control signals, address signals, and data signals), and generates signals for controlling the individual memories based on these input signals. The embedded memory I / F is connected to each functional block, receives memory access signals (e.g., control signals, address signals, and data signals), and generates modified memory access signals for controlling the embedded memories based on these input signals. The separate memory I / F and the embedded memory I / F may be designed within the memory control block of the ASIC to provide a memory control structure that can be flexibly applied to both the individual memories and the embedded memories.

[0167] Additionally, an application-specific integrated circuit (ASIC) for an artificial neural network (ANN) may be configured to include a plurality of neurons arranged in an array and a plurality of synaptic circuits, each neuron including a register, a microprocessor, and at least one input, and each synaptic circuit including a memory for storing synaptic weights. Each neuron of the ASIC may be connected to at least one other neuron through one of the plurality of synaptic circuits.

[0168] Although the present disclosure has been described above as being generally implemented by a computing device, those skilled in the art will appreciate that the present disclosure may also be implemented in combination with computer-executable instructions and / or other program modules that may be executed on one or more computers and / or as a combination of hardware and software.

[0169] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0170] Those skilled in the art will appreciate that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, various forms of programs or design code (referred to herein, for convenience, as software), or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0171] The various embodiments presented herein can be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term article of manufacture includes a computer program, carrier, or media accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Furthermore, various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0172] It should be understood that the specific order or hierarchy of steps in the presented processes is merely an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but are not intended to be limited to the specific order or hierarchy presented.

[0173] FIG. 2 illustrates an example of a block diagram of a computing system device (200) that implements a service for predicting the EC number of a protein according to one embodiment of the present disclosure.

[0174] The various operations of the system and method for predicting the EC number of a protein of the present disclosure may be performed by any suitable means capable of performing the corresponding functions. Such means may include various hardware, software components, modules, and combinations thereof, including but not limited to circuits, processors, application-specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0175] Referring to FIG. 2, a computing system (200) implementing an artificial neural network of the present disclosure may include a transceiver (210), a memory (220), a database (230), and a processor (240). However, not all of the components illustrated in FIG. 2 are essential components of the computing system device (200). The computing system device (200) may be implemented with more components than the components illustrated in FIG. 2, or may be implemented with fewer components than the components illustrated in FIG. 2. In addition, the transceiver (210), the memory (220), and the processor (240) may be implemented in the form of a single chip.

[0176] In one embodiment, the transceiver (210) may communicate with a terminal or other electronic device connected to the computing system device (200) via wired or wireless communication. For example, the transceiver (210) may obtain protein amino acid sequence information, protein structure information, or protein representations generated using an artificial neural network from the other electronic device.

[0177] Various types of data, such as programs and files, such as applications, can be installed and stored in the memory (220). The processor (240) can access and use data stored in the memory (220), or store new data in the memory (220). In addition, the memory (220) can store one or more instructions. The processor (240) can execute one or more instructions stored in the memory.

[0178] The processor (240) controls the overall operation of the computing system device (200). Here, the processor (240) may be composed of at least one or a plurality of processors electrically connected among a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit, a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array, controllers, micro-controllers, microprocessors and / or other electrical units for performing functions.

[0179] The processor (240) may control other components included in the computing system device (200) to perform operations for operating the computing system device (200). For example, the processor (240) may obtain protein amino acid sequence information, obtain protein representations using the neural network, derive a contrastive loss function from the protein representations, and improve one or more values ​​or parameters of one or more neural networks among the encoder neural network and the projection neural network based on the contrastive loss function.

[0180] The database (230) may store various types of learning data for training a learning model. Furthermore, the database (230) may store protein amino acid sequence information, protein structure information, simulation result information, and the like. In various embodiments, the database may also store output data generated by the learning model. While FIG. 2 illustrates the computing system device (200) as including the database (230), the database (230) may be provided externally to the device. In this case, the database (230) may be connected to the computing system device (200) via wired or wireless connections.

[0181] Additionally, the learning model of the present disclosure may be implemented outside the computing system device (200) (e.g., cloud-based) or may be included within the computing system device (200).

[0182] An embodiment of the present disclosure may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically contains computer-readable instructions, data structures, or program modules, and includes any information delivery media.

[0183] Throughout the present disclosure, devices may include, but are not limited to, servers, smartphones, tablet PCs, PCs, TVs, smart TVs, mobile phones, personal digital assistants (PDAs), speakers, laptops, media players, microservers, e-book object recognition devices, digital broadcasting object recognition devices, kiosks, MP3 players, digital cameras, robot vacuum cleaners, home appliances, other mobile or non-mobile computing devices, watches, glasses, hair bands, and rings having communication and data processing capabilities.

[0184] 8. Application of the system

[0185] The system of the present disclosure can be used to predict the EC number of an enzyme from the amino acid sequence of a protein, thereby predicting the biological function of the enzyme. Specifically, when the amino acid sequence of a protein is input into the system of the present disclosure, the artificial intelligence model of the system of the present disclosure can predict the EC number that best matches the input amino acid sequence of the protein. Since each EC number is defined by a unique biological function, the biological function of the protein can be predicted based on the output EC number. The system can be used to elucidate the function of an unknown protein amino acid sequence.

[0186] In addition, the system of the present disclosure can be used to predict the amino acid sequence of an enzyme having a biological function based on the EC number of the enzyme. Specifically, when an EC number is input into the system of the present disclosure, the artificial intelligence model of the system of the present disclosure can predict and output the amino acid sequence of a protein having a function corresponding to the input EC number. For example, when EC numbers 2, 3, 1, and 9 are input, the system can predict and output the amino acid sequence of a protein having one or more acetyl acetyl-CoA C-acetyltransferase functions. The artificial intelligence model of the system is a model learned through learning data, and the learning data can include the amino acid sequence of an enzyme and the corresponding EC number. The system can also be used to obtain the amino acid sequence of a novel enzyme that can be a drug candidate.

[0187] The above description of the present disclosure is provided for illustrative purposes only, and those skilled in the art will readily appreciate that the present disclosure can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0188] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

[0189] Example

[0190] 1. Apply hierarchical multi-label contrast to levels 3 and 4.

[0191] The performance of hierarchical multi-label contrast applied to levels 3 and 4 is shown in Figures 3a to 3c. When hierarchical multi-label contrast was applied only to level 4, it performed identically to vanilla supervised contrastive learning. In contrast, when hierarchical multi-label contrast was applied only to level 3, it performed superiorly for high-level EC numbers, but did not perform well in predicting EC numbers at level 4.

[0192] 2. Apply hierarchical multi-label contrast scaling to all levels.

[0193] Hierarchical multi-label contrast was extended to encompass all levels of the EC number hierarchy. Weights at each level were selected based on an incremental grid search from lower to higher levels, and the results are shown in Figure 4. These results demonstrate that incrementally incorporating hierarchical structures into the hierarchical multi-label contrast loss can improve performance at higher levels while maintaining performance at lower levels.

[0194] 3. Predicting EC numbers not seen in the training data set

[0195] While predicting an EC number of level 4 for a completely new enzyme for which no EC number has been previously defined is impossible, predicting EC numbers at higher levels, such as level 3, is possible. Therefore, accurately predicting high-level EC numbers is crucial for increasing the reliability of the model's predictions for novel proteins. Figure 5 shows the performance at level l, one level lower, for proteins with labels at a lower level (l + 1) that were not seen in the training dataset. The results show that incorporating higher levels into the hierarchical multi-label contrastive loss significantly improves performance.

[0196] 4. Comparative evaluation with benchmarking models

[0197] The results of a comparative evaluation of the performance of three benchmark models are shown in Figures 6a and 6b. Overall, model levels 1-4 and 2-4 of the present disclosure achieved state-of-the-art F1 scores across different levels on the New-392 and Price-149 datasets. Furthermore, model level 4 demonstrated a better F1 score than CLEAN.

[0198] 5. Robustness

[0199] The number of proteins with matching predictions across all 5-fold models for the New-392 and Price-149 datasets is counted and shown in Figure 7. These results demonstrate the robustness of the models. The model of this disclosure achieved the best results, and the results increased progressively as more hierarchical levels were applied. This demonstrates that hierarchical contrastive learning improves robustness.

[0200] 6. Case studies on the function of novel enzymes

[0201] The case study of this disclosure demonstrates the effectiveness of incorporating EC number hierarchy in predicting the function of a novel enzyme. A0A1D8PH52 in the NEW-392 dataset belongs to class 2.3.1.9 (acetyl-CoA C-acetyltransferase), but was misclassified as class 2.3.3.10 (hydroxymethylglutaryl-CoA synthase) when initial hierarchical multi-label contrast loss was applied only to class 4. Although the model accurately predicted EC numbers for class 2, it did not for class 3. By considering the EC number hierarchy according to the method of this disclosure, the enzyme was correctly identified as class 2.3.1.9 (Figure 8).

[0202] This paper demonstrates the effectiveness of hierarchical contrastive learning in EC number prediction. Leveraging the hierarchical nature of EC numbers improves model performance and robustness compared to state-of-the-art models, particularly for high-level EC numbers and previously unseen EC numbers. These results demonstrate the potential of hierarchical contrastive learning to enhance the performance of these models, providing a robust and scalable solution for accurate prediction of enzyme function.

Claims

1. In a protein function prediction system, a memory that stores one or more instructions, and At least one executing one or more of the commands stored in the memory including processors, The actions performed by one or more of the above commands are A step of obtaining protein amino acid sequence information, and A step of predicting the EC number of a protein having the protein amino acid sequence information using at least one artificial intelligence model learned by hierarchical contrastive learning, A protein function prediction system characterized in that the hierarchical contrastive learning is learned using a hierarchical multi-label contrastive loss.

2. A protein function prediction system according to claim 1, wherein the hierarchical contrastive learning performs hierarchical multi-label contrastive learning for one or more EC number hierarchical levels.

3. A protein function prediction system according to claim 1, wherein the hierarchical contrastive learning is learned using a training data set including training data having no information for one or more EC number hierarchical levels, and the learned model predicts the EC number of a level one level higher than the EC number hierarchical level having no information.

4. In the first paragraph, the hierarchical contrastive learning (i) When calculating the loss at the lowest level, all dimension representations are used, and as you go up the level, some dimensions of the representation used at the lower level are used to calculate the loss, or (ii) When calculating the loss for each level, only some of the dimension representations are used, and some of the dimensions used for each level are used in calculating the loss for other levels, or (iii) Use them separately so that the dimensions used for each level do not overlap, or (iv) Set the number of learnable coefficient vectors of the same size as the expression vector to the same number as the number of levels, and use the product of the expression and the coefficient vector of each level when calculating the loss, or (v) Perform it by placing as many multilayer perceptrons as the number of levels, or A protein function prediction system, which is performed including steps performed through the above combination.

5. A protein function prediction system according to claim 1, wherein the hierarchical contrastive learning obtains a feature vector of a protein amino acid sequence using a trained protein language model.

6. A protein function prediction system according to claim 1, wherein the hierarchical contrastive learning is performed by converting a feature vector of the amino acid sequence of the protein into a representation vector using a multilayer perceptron having three hidden layers.

7. A protein function prediction system, comprising a step of inferring based on the distance between the EC number embedding and the query protein embedding in the first paragraph, wherein the distance is a Euclidean distance.

8. A method for predicting protein function performed by at least one processor, the method comprising a step of predicting protein function using a protein amino acid sequence, The above prediction is, A step of obtaining protein amino acid sequence information, and A step of predicting the EC number of a protein having the protein amino acid sequence information using at least one artificial intelligence model learned by hierarchical contrastive learning, A protein function prediction method characterized in that the hierarchical contrastive learning is learned using a hierarchical multi-label contrastive loss.

9. A method for predicting protein function in accordance with claim 8, wherein the hierarchical contrastive learning performs hierarchical multi-label contrastive learning for one or more EC number hierarchical levels.

10. A method for predicting protein function, characterized in that in the 8th paragraph, the hierarchical contrastive learning is learned using a training data set including training data having no information for one or more EC number hierarchical levels, and the learned model predicts the EC number of a level one level higher than the EC number hierarchical level having no information.

11. In the 8th paragraph, the hierarchical contrastive learning (i) When calculating the loss at the lowest level, all dimension representations are used, and as you go up the level, some dimensions of the representation used at the lower level are used to calculate the loss, or (ii) When calculating the loss for each level, only some of the dimension representations are used, and some of the dimensions used for each level are used in calculating the loss for other levels, or (iii) Use them separately so that the dimensions used for each level do not overlap, or (iv) Set the number of learnable coefficient vectors of the same size as the expression vector to the same number as the number of levels, and use the product of the expression and the coefficient vector of each level when calculating the loss, or (v) Perform it by placing as many multilayer perceptrons as the number of levels, or A method for predicting protein function, which is performed including steps performed through the above combination.

12. A method for predicting protein function in accordance with claim 8, wherein the hierarchical contrastive learning obtains a feature vector of a protein amino acid sequence using a trained protein language model.

13. A method for predicting protein function in accordance with claim 8, wherein the hierarchical contrastive learning is performed by converting a feature vector of the amino acid sequence of the protein into a representation vector using a multilayer perceptron having three hidden layers.

14. A method for predicting protein function, comprising a step of inferring based on the distance between the EC number embedding and the query protein embedding in the 8th paragraph, wherein the distance is a Euclidean distance.

15. A program stored on a computer-readable recording medium that causes a computer to execute the method of any one of clauses 8 to 14.

16. A computerized method for predicting protein function, A step of inputting protein amino acid sequence information into an artificial intelligence model executed by a processor, and A step of predicting the EC number of a protein having the protein amino acid sequence information using an artificial intelligence model learned by hierarchical contrastive learning executed by at least one processor, The above hierarchical contrastive learning is characterized in that it learns using a hierarchical multi-label contrastive loss. Methods for predicting protein function.

17. A method for predicting protein function in claim 16, wherein the hierarchical contrastive learning performs hierarchical multi-label contrastive learning for one or more EC number hierarchical levels.

18. A method for predicting protein function, characterized in that in paragraph 16, the hierarchical contrastive learning is learned using a training data set including training data having no information for one or more EC number hierarchical levels, and the learned model predicts the EC number of a level one level higher than the EC number hierarchical level having no information.

19. In paragraph 16, the hierarchical contrastive learning (i) When calculating the loss at the lowest level, all dimension representations are used, and as you go up the level, some dimensions of the representation used at the lower level are used to calculate the loss, or (ii) When calculating the loss for each level, only some of the dimension representations are used, and some of the dimensions used for each level are used in calculating the loss for other levels, or (iii) Use them separately so that the dimensions used for each level do not overlap, or (iv) Set the number of learnable coefficient vectors of the same size as the expression vector to the same number as the number of levels, and use the product of the expression and the coefficient vector of each level when calculating the loss, or (v) Perform it by placing as many multilayer perceptrons as the number of levels, or A method for predicting protein function, which is performed including steps performed through the above combination.

20. A method for predicting protein function in claim 16, wherein the hierarchical contrastive learning obtains a feature vector of a protein amino acid sequence using a trained protein language model.

Images