System and method for predicting protein structure

Abstract

The present invention relates to a system and method for predicting a protein structure, wherein protein data including amino acid sequence information of one or more proteins is converted into embeddings, a first model and a second model each generate logits from the embeddings, a softmax function using a temperature parameter is applied to the logits respectively generated by the first model and the second model, to generate prediction distributions, a loss function is calculated by using the prediction distributions, and learning is carried out such that the loss function decreases.

Description

Protein Structure Prediction System and Method

[0001] The present disclosure relates to a probabilistic circuit-based protein structure prediction system and method capable of increasing the accuracy and efficiency of protein structure prediction and performing various probabilistic inferences.

[0002] It has long been known that proteins are the core molecules of living organisms that serve as the source of various life phenomena, yet their structure and function have not been precisely elucidated until recently. With the conclusion of the Human Genome Project and the rapid advancement of gene sequencing technology, a significant number of amino acid sequences constituting proteins have been revealed, but aspects such as the structure, physical properties, and function of proteins have not yet been fully elucidated.

[0003] Recently, protein structure prediction technology utilizing artificial intelligence has advanced dramatically. In particular, AlphaFold, developed by DeepMind, achieved significant prediction accuracy by combining diffusion models. While diffusion models probabilistically model the protein folding process, they have the disadvantage of very high computational costs due to the need for iterative sampling and inference processes. Furthermore, generative models such as diffusion models or variant autoencoders excel at generating overall structures, but they have structural limitations in efficiently processing complex probabilistic queries, such as predicting conditional structures while fixing specific amino acid sequences or inferring structural uncertainty under specific environmental variables.

[0004] Accordingly, the inventors intend to provide a protein structure prediction system and method through the present disclosure that combines a probabilistic circuit module with a protein structure prediction model using a conventional diffusion model to alleviate the difficulty of the learning process and rapidly obtain a high-performance probabilistic circuit model.

[0005] The problem addressed by the present disclosure is to provide a protein structure prediction system and method capable of precise structure prediction and uncertainty inference based on partial sequences or environmental conditions, which are difficult to perform with conventional protein structure prediction models.

[0006] To solve the above problem, the present disclosure provides a protein structure prediction system and method.

[0007] The present disclosure comprises a memory for storing one or more instructions; and

[0008] At least one that executes the one or more instructions stored in the memory.

[0009] Includes processors,

[0010] The operation performed by the above one or more instructions is

[0011] A step of converting protein data containing one or more protein amino acid sequence information into an embedding,

[0012] The first model and the second model each generate a logit from the embedding,

[0013] A step of generating a prediction distribution by applying a softmax function using temperature parameters to the logits generated by the first model and the second model, respectively.

[0014] A step of using a prediction distribution generated through the first model as a soft target and obtaining a soft loss function by comparing it with a prediction distribution generated through the second model, and

[0015] A protein structure prediction system is provided that includes the step of obtaining a loss function using the soft loss function and performing learning to reduce the loss function.

[0016] The present disclosure provides a protein structure prediction system in which the first model performs protein structure prediction using a diffusion model.

[0017] The present disclosure provides a protein structure prediction system in which the second model is a stochastic circuit model.

[0018] The present disclosure provides a protein structure prediction system in which the probabilistic circuit model is a sum-product network or a probabilistic sentence decision diagram.

[0019] The present disclosure provides a protein structure prediction system in which the loss function is further calculated by using a loss function obtained by calculating the difference between the prediction of a second model and the actual correct label.

[0020] The present disclosure provides a protein structure prediction system that performs learning by additionally using one method selected from expectation maximization or stochastic gradient descent in the step of performing learning to reduce the loss function.

[0021] The present disclosure provides a protein structure prediction system that calculates a result value for predicting a protein structure using a second model that has been trained, and calculates a result value for one or more selected from a group consisting of confidence, probability distribution, and uncertainty for the prediction.

[0022] The present disclosure provides a method for predicting a protein structure performed by at least one processor.

[0023] The present disclosure comprises the step of converting protein data containing one or more protein amino acid sequence information into an embedding,

[0024] The first model and the second model each generate a logit from the embedding,

[0025] A step of generating a prediction distribution by applying a softmax function using temperature parameters to the logits generated by the first model and the second model, respectively.

[0026] A step of using a prediction distribution generated through the first model as a soft target and obtaining a soft loss function by comparing it with a prediction distribution generated through the second model, and

[0027] A method for predicting protein structure is provided, comprising the step of obtaining a loss function using the soft loss function and performing training to reduce the loss function.

[0028] The present disclosure provides a protein structure prediction method in which the first model performs protein structure prediction using a diffusion model.

[0029] The present disclosure provides a method for predicting protein structure in which the second model is a stochastic circuit model.

[0030] The present disclosure provides a method for predicting protein structure in which the stochastic circuit model is a sum-product network or a stochastic sentence decision diagram.

[0031] The present disclosure provides a protein structure prediction method in which the loss function is further calculated by using a loss function obtained by calculating the difference between the prediction of a second model and the actual correct label.

[0032] The present disclosure provides a protein structure prediction method that, in the step of performing learning to reduce the loss function, additionally uses one method selected from expectation maximization or stochastic gradient descent to perform learning.

[0033] The present disclosure provides a protein structure prediction method that calculates a result value for predicting a protein structure using a second model that has been trained, and calculates a result value for one or more selected from a group consisting of confidence, probability distribution, and uncertainty for the prediction.

[0034] The present disclosure provides a program stored on a computer-readable recording medium to execute the above method on a computer.

[0035] By using the protein structure prediction system and method of the present disclosure, the difficulty of learning large-scale protein data sets can be alleviated compared to conventional technology.

[0036] In addition, by using the protein structure prediction system and method of the present disclosure, various probability queries that were difficult to implement in conventional technologies such as diffusion models can be performed, and immediate and flexible prediction and inference tasks such as conditional probability, marginalization, and sampling can be performed.

[0037] Furthermore, by using the protein structure prediction system and method of the present disclosure, rapid and efficient protein structure prediction and inference can be performed with only a single or a small number of operations without the iterative sampling process of the diffusion model.

[0038] FIG. 1 is a flowchart of a protein structure prediction system and method according to one embodiment of the present disclosure.

[0039] FIG. 2 is a schematic diagram of a protein structure prediction system and method according to one embodiment of the present disclosure.

[0040] FIG. 3 is a structural diagram of a probabilistic circuit model used as a second model according to one embodiment of the present disclosure.

[0041] FIG. 4 is a block diagram of an apparatus capable of implementing a protein structure prediction system and method according to one embodiment of the present disclosure.

[0042] Figure 5 shows an exemplary on-premise full stack structure.

[0043] To clarify the technical concept of the present disclosure, embodiments of the present disclosure will be described in detail with reference to the attached drawings. In describing the present disclosure, detailed descriptions of related known functions or components will be omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of the present disclosure. Components having substantially the same function or configuration among the drawings have been assigned the same reference numerals and symbols as much as possible, even if they are shown in different drawings. For convenience of explanation, devices and methods are described together where necessary. Each operation of the present disclosure does not necessarily need to be performed in the order described and may be performed in parallel, selectively, or individually.

[0044] The terms used in the embodiments of this disclosure have been selected to be as widely used and general as possible, taking into account the functions of this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the description of the relevant embodiments. Therefore, terms used in this specification should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0045] Throughout this disclosure, singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms such as "comprising" or "having" are intended to specify the existence of features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. That is, throughout this disclosure, when a part is described as "comprising" a certain component, it means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0046] Expressions such as "at least one" modify the entire list of components and do not modify the components of the list individually. For example, "at least one of A, B, and C" and "at least one of A, B, or C" refer to only A, only B, only C, both A and B, both B and C, both A and C, all of A, B, and C, or any combination thereof.

[0047] Additionally, terms such as “part,” “module,” etc., as described in this disclosure refer to a unit that processes at least one function or operation, and may be implemented in hardware or software, or a combination of hardware and software.

[0048] Throughout the entire disclosure, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other elements interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0049] Throughout this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware. Instead, in some situations, the expression “system configured to” may mean that the system is “capable of” in conjunction with other devices or components. For example, the phrase “processor configured to perform A, B, and C” may mean a dedicated processor for performing the said operations (e.g., an embedded processor) or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in memory.

[0050] The functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0051] Artificial intelligence is a field of computer science and information technology that studies methods to enable computers to perform thinking, learning, and self-development tasks typically accomplished by human intelligence; it refers to the ability of computers to mimic intelligent human behavior. Furthermore, artificial intelligence does not exist in isolation but is closely related, directly or indirectly, to many other areas of computer science. Particularly in the modern era, there is active research being conducted across various fields of information technology to introduce AI elements and utilize them for problem-solving.

[0052] Machine learning is a field of artificial intelligence that enables computers to learn without explicit programming. Specifically, machine learning can be defined as a technology that studies and builds systems and algorithms capable of learning, making predictions, and improving their own performance based on empirical data. Rather than executing strictly defined static program commands, machine learning algorithms adopt an approach of constructing specific models to derive predictions or decisions based on input data. The term 'machine learning' may be used interchangeably with 'machine learning'.

[0053] Many machine learning algorithms have been developed to address the question of how to classify data in machine learning. Representative examples of these algorithms include Decision Trees, Bayesian Networks, Support Vector Machines (SVM), and Artificial Neural Networks (ANN).

[0054] A decision tree is an analytical method that performs classification and prediction by plotting decision rules in a tree structure. A Bayesian network is a model that represents the probabilistic relationships (conditional independence) between multiple variables in a graph structure. Bayesian networks are suitable for data mining through unsupervised learning.

[0055] Support Vector Machines are supervised learning models for pattern recognition and data analysis, primarily used for classification and regression analysis. Artificial neural networks model the operating principles of biological neurons and the relationships between them; they are information processing systems in which multiple neurons, referred to as nodes or processing elements, are connected in a layered structure.

[0056] Artificial neural networks are models used in machine learning, serving as statistical learning algorithms in machine learning and cognitive science that draw inspiration from biological neural networks (particularly the brain within the animal central nervous system). Specifically, an artificial neural network can refer to a model in which artificial neurons (nodes), forming a network through the connection of synapses, change the strength of these connections through learning to possess problem-solving capabilities. The term 'artificial neural network' may be used interchangeably with 'neural network'.

[0057] An artificial neural network may include multiple layers, and each layer may include multiple neurons. Additionally, an artificial neural network may include synapses connecting neurons. An artificial neural network can generally be defined by the following three factors: the connection patterns between neurons in different layers, a learning process that updates the weights of the connections, and an activation function that generates an output value from a weighted sum of inputs received from the previous layer.

[0058] Artificial neural networks may include, but are not limited to, network models such as Deep Neural Networks (DNN), Recurrent Neural Networks (RNN), Bidirectional Recurrent Deep Neural Networks (BRDNN), Multilayer Perceptrons (MLP), and Convolutional Neural Networks (CNN). In this specification, the term 'layer' may be used interchangeably with the term 'layer'.

[0059] Artificial neural networks are classified into single-layer neural networks and multi-layer neural networks depending on the number of layers. A typical single-layer neural network consists of an input layer and an output layer. Additionally, a typical multi-layer neural network consists of an input layer, one or more hidden layers, and an output layer.

[0060] The input layer is a layer that receives external data, and the number of neurons in the input layer is equal to the number of input variables. The hidden layer is located between the input layer and the output layer, receives signals from the input layer, extracts features, and transmits them to the output layer. The output layer receives signals from the hidden layer and outputs an output value based on the received signals. Input signals between neurons are multiplied by their respective connection strengths (weights) and then summed; if this sum is greater than the neuron's threshold, the neuron is activated and outputs the value obtained through the activation function.

[0061] Meanwhile, a deep neural network containing multiple hidden layers between the input layer and the output layer can be a representative artificial neural network that implements deep learning, a type of machine learning technique. Meanwhile, the term 'deep learning' may be used interchangeably with the term 'deep learning'.

[0062] The machine learning workflow consists of a series of processes involving collecting data for learning and validation, modeling, and training the model, and may include the processes of collecting training data, checking and exploring data, data preprocessing and cleaning, modeling, and training.

[0063] The present disclosure provides a system and method for predicting protein structures quickly and efficiently as follows.

[0064] 1. Protein Data Acquisition

[0065] Protein data that can be used for predicting protein structure in the system and method of the present disclosure includes, without limitation, any data that can be utilized for predicting protein structure. For example, protein amino acid sequence information, multiple sequence alignment information, contact maps, secondary structure prediction information, etc., may be used. According to one embodiment of the present disclosure, the protein data includes one or more protein amino acid sequence information.

[0066] 2. Inspection and Exploration of Training Data

[0067] Once training data for learning an artificial intelligence model is collected, the collected training data can be examined and explored regarding its structure, noise data, and data cleaning methods for machine learning applications.

[0068] This stage of data inspection and exploration is called Exploratory Data Analysis (EDA), which can be described as the process of observing and understanding collected data from various angles. Before training the data, independent variables, dependent variables, variable types, and data types are examined using visualizations such as graphs and statistical tests, allowing the characteristics of the data and inherent structural relationships to be identified in advance. Through this EDA, examining the distribution and values ​​of the data enables a better understanding of the phenomena represented by the data and the discovery of potential problems. Furthermore, by examining the data from various angles, diverse patterns that might not have been identified during the problem definition stage can be discovered, allowing for the modification of existing hypotheses or the formulation of new ones. Exploratory data analysis can broadly encompass the process of searching for data outliers and analyzing the relationships between data attributes.

[0069] The process of detecting outliers involves verifying whether they exist in the data and can include sampling methods, statistical methods, and visualization methods. Sampling methods involve drawing random samples from the data to identify overall trends and anomalies in the data values. Statistical methods may utilize summary statistics, such as the mean, median, and mode to identify the center of the data, or range and variance to check the dispersion. Visualization methods utilize probability density functions, histograms, dot plots, word clouds, time series charts, and maps to determine which statistical indicators are appropriate for the individual attributes of the collected data. However, when using statistical indicators, caution should be exercised as the mean reflects all data values ​​within a set and is therefore affected by outliers, whereas the median uses only the single value in the middle, allowing for representative results even in the presence of outliers.

[0070] The process of analyzing relationships between data attributes involves identifying combinations of attributes within the data that possess meaningful correlations. Relationship analysis can be conducted differently depending on the combination of attributes between qualitative attributes (Categorical Variables; Qualitative), which cannot be expressed numerically but can be arbitrarily quantified, and quantitative attributes (Numeric Variables; Quantitative), which can be quantified. Categorical-categorical relationships can display the number of values ​​corresponding to each pair of attribute values ​​using cross-tabulation tables or mosaic plots; Numeric-categorical relationships can be visually represented through box plots or by observing statistical values ​​by category (mean, median, etc.); and Numeric-numeric relationships can analyze the association between two attributes using correlation coefficients. It can be confirmed that a correlation coefficient of -1 indicates a negative correlation where the two attributes change in opposite directions, 0 indicates no correlation, and 1 indicates a positive correlation where the two attributes always change in the same direction. The relationship between two attributes with a correlation coefficient can also exhibit various aspects, which can be visually represented using a scatter plot.

[0071] 3. Preprocessing of Training Data

[0072] Data that has completed inspection and exploration undergoes data preprocessing to transform it into a format suitable for machine learning training models. Data preprocessing involves cleaning the data and converting it into a form that the model can understand; it generally includes handling missing data, outlier removal, data scaling, categorical data encoding, feature selection and extraction, and data transformation. The detailed processes of data preprocessing may be performed in whole or in part selectively, and a separate machine learning model may be used for this purpose.

[0073] Handling Missing Data is the process of handling missing values ​​when they exist in the data; these values ​​can be displayed as NaN (Not a Number) or empty, or deleted. Filling in or deleting missing values ​​improves data completeness, and values ​​such as the mean, median, or mode may be used when filling in missing values.

[0074] Outlier removal is the process of eliminating outliers, which are values ​​that deviate from typical data patterns. Since outliers can degrade model performance, they must be removed or replaced; this involves identifying outliers and deleting the corresponding rows or columns or replacing them with other values.

[0075] Data scaling is the process of adjusting the size of data; through data scaling, the range of the data is adjusted, which can improve model performance or accelerate convergence. Data scaling allows data characteristics to be aligned within a similar range, and generally, standardization and normalization can be applied.

[0076] Categorical Data Encoding is the process of converting categorical variables, which are represented as string or integer values ​​and cannot be directly input into a model, into a numeric type that can be input. Generally, one-hot encoding or label encoding can be used to convert categorical variables into numeric types.

[0077] Feature selection and extraction is intended to improve the performance of a model by selecting the most useful features for model training or extracting new features. Through this process, the complexity of the model can be reduced and overfitting can be prevented.

[0078] Data transformation involves converting data to extract new information or enable a model to understand it better, and may include the tokenization of text data or the preprocessing of image data. Through data transformation, model performance can be improved by extracting useful features from original data or converting data into an appropriate format.

[0079] Through data preprocessing as described above, it is possible to achieve the effects of improving the performance and ensuring the stability of machine learning models.

[0080] Meanwhile, if the collected data has not been preprocessed according to the requirements, tokenization, cleaning, and normalization can be performed to suit the intended use of the data.

[0081] In order for a computer to understand and process text, it must be appropriately converted into numbers. Since the performance of natural language processing varies significantly depending on how words are represented, many techniques have been proposed to quantify words. Currently, word embedding methods, which vectorize each word through artificial neural network learning, are frequently used.

[0082] 4. Structure of the protein structure prediction model

[0083] The present disclosure relates to a protein structure prediction system and method, which may operate in a manner similar to FIGS. 1 and FIGS. 2 in one embodiment.

[0084] Data such as protein amino acid sequences and / or multiple sequence alignment information input into the system of the present disclosure is converted into embeddings through a preprocessing process via a data preprocessing unit (101).

[0085] The first model of the present disclosure performs protein structure prediction from the embeddings to generate logits (102). The first model is a large artificial neural network that is pre-trained and exhibits high performance in protein structure prediction. The artificial neural network that can be used as the first model in the present disclosure is not particularly limited, but a representative example is the diffusion model of AlphaFold 3, which generates a 3D structure prediction value of a protein and a probability distribution of the structure based on input embeddings, and the system and method of the present disclosure can also be applied to other models introduced after the filing date of the present disclosure.

[0086] In the system and method of the present disclosure, the second model performs learning by mimicking the first model. The second model may be a stochastic circuit model such as FIG. 3. The stochastic circuit model that can be used in the present disclosure is not particularly limited, but may be implemented as a sum-product network (SPN) or a stochastic sentence decision diagram (PSDD), etc. The second model generates a logit, which is a denormalized prediction score vector, from an embedding generated by preprocessing protein data (102).

[0087] In the present disclosure, the learning unit performs learning so that the protein structure prediction of the second model mimics the protein structure prediction of the first model as much as possible. The learning unit of the present disclosure generates prediction distributions by applying a softmax function using temperature parameters to the logits output by the first model and the second model (103).

[0088] In the present disclosure, a soft target refers to a probability distribution obtained by passing a softmax function, which applies a temperature parameter (T) to the logit, which is the output value of an artificial intelligence model. This is used for learning through knowledge distillation and plays an important role in transferring knowledge from the first model to the second model in the present disclosure. According to one embodiment of the present disclosure, the prediction distribution can be obtained through the function of the following equation (1).

[0089] [Equation (1)]

[0090]

[0091] In the above equation (1), Q(i) is the probability value for the i-th class of the prediction distribution output by the model, zi is the value corresponding to the i-th class in the model's logit vector Z, T is the temperature parameter, and K is the total number of classes that the model must classify.

[0092] The learning unit sets the prediction distribution generated through the first model as a soft target and calculates a soft loss function by comparing it with the prediction distribution generated through the second model (104). The soft loss function is calculated by determining the difference between the prediction distribution of the first model and the prediction distribution of the second model, and represents the similarity between the prediction distributions generated by the two models. The loss function used in the system and method of the present disclosure is not particularly limited, but, for example, a loss function such as mean squared error or Kullback-Leibler divergence can be used to minimize the difference between the prediction distributions of the first model and the second model.

[0093] In the present disclosure, a loss function may be additionally utilized to further improve the learning performance of protein structure prediction. The loss function additionally utilized in the present disclosure is a loss function (i.e., a hard loss function) obtained using a hard target, which is a ground truth label in the form of one-hot encoding. The hard loss function calculates the difference between the prediction of the second model and the actual ground truth label. In the present disclosure, a soft loss function and a hard loss function may be used in combination; for example, learning may be performed by determining the final loss function as the sum of the values ​​obtained by weighting the soft loss function and the hard loss function, respectively.

[0094] In the present disclosure, parameters at each node (sum node, product node, etc.) of a stochastic circuit can be updated using an optimization algorithm such as expectation maximization or stochastic gradient descent to minimize the loss function.

[0095] The system and method of the present disclosure perform learning to reduce the loss function (105), and through this process, the second model can effectively simulate the probability distribution predicted by the first model.

[0096] A second model that has completed training according to the system and method of the present disclosure performs inference and prediction of protein structure with high performance while being lighter and more efficient than the first model.

[0097] After training is complete, the second model can flexibly perform various prediction and inference tasks, such as conditional probability (e.g., predicting the overall protein structure when the amino acid sequence of a specific region is fixed), marginalization (e.g., predicting the protein structure when some amino acid sequences are uncertain), and sampling (e.g., simulating a new protein structure). Furthermore, by utilizing the structural characteristics of stochastic circuits, it can easily calculate uncertainty information such as confidence, probability distribution, variance, and likelihood regarding the prediction. Additionally, it can immediately generate results regarding probability distribution, protein structure prediction, and uncertainty with only a single or a small number of operations, without the iterative sampling process of the diffusion module in the first model.

[0098] The second model, once trained, can be used in parallel or sequentially with the first model, and the reliability of the final structure prediction can be further improved by utilizing ensemble techniques.

[0099] The degrees of freedom for probabilistic inference can be increased through the system and method of the present invention. While complex conditional and marginalization operations are difficult in conventional diffusion models or modified autoencoder models, immediate and intuitive inference is possible through the system and method of the present disclosure. For example, mutations can be assumed, or the structural stability of a protein under specific environmental conditions, such as pH, can be evaluated by marginalizing them.

[0100] Efficient learning and inference are possible through the system and method of the present invention. By receiving knowledge already learned in the first model into the second model, the difficult initial learning problem of the circuit module can be alleviated. As a result, protein structure prediction and uncertainty analysis can be performed with a single or small number of circuit operations without the burden of iterative sampling in the diffusion model.

[0101] Precise uncertainty estimation is possible through the system and method of the present invention. By receiving knowledge already learned in the first model into the second model, the difficult initial learning problem of the circuit module can be alleviated. As a result, protein structure prediction and uncertainty analysis can be performed with a single or a small number of circuit operations without the burden of iterative sampling in the diffusion model.

[0102] The system and method of the present invention have the advantage of being able to calculate structural reliability, which is of significant importance in new drug development.

[0103] The system of the present invention has high utility in various application fields, such as new drug development and life science research, through this reasoning ability.

[0104] MoE (Mixture of Experts) Architecture

[0105] In one embodiment of the present disclosure, a system for predicting phenotypes may be performed by utilizing a model architecture such as MoE. Here, MoE may refer to an architecture of a machine learning model that solves complex problems by combining multiple expert models.

[0106] Such MoE may include expert models, which are multiple small networks designed to learn different parts and / or different features of a given data and perform data processing operations accordingly, and a gating network that evaluates the performance of each expert model and determines which expert model is most suitable for assigning a specific task based on the given data based on this evaluation.

[0107] Thus, according to the MoE architecture, a gating network that acquires predetermined input data determines probabilistic or deterministic task assignments for each expert model, and the selected expert models perform their respective tasks and return the results, thereby enabling data processing for a specific task.

[0108] According to one embodiment of the present disclosure, a MoE model used may refer to a specific MoE model implemented according to a common method known in the art. For example, the MoE model may include a Switch Transformer, Conditional Computation in Neural Networks, Sparse Mixture of Experts, and / or a Megatron-LM.

[0109] Additionally, in one embodiment of the present disclosure, a MoE model based on the combination of a plurality of Specialized Models (SM) and Routers (Gating Network, RT) may be included, and a MoE model based on domain-specific Specialized Models may also be included.

[0110] By utilizing such MoE, the overall efficiency and performance of the phenotype prediction system of the present disclosure can be enhanced by activating only specific parts and concentrating computational resources in cases such as handling complex tasks or large datasets.

[0111] 5. Computing System Device

[0112] Embodiments of the present disclosure may be implemented as application-specific integrated circuits (ASICs) designed to suit specific applications and special functions of devices.

[0113] Custom integrated circuits are also referred to as custom semiconductors. Unlike standard semiconductors, which have fixed specifications and can be applied to any electronic product or application as long as certain requirements are met, custom semiconductors are used for specific products or functions and are integrated circuits manufactured by semiconductor companies to meet specific orders. In other words, custom semiconductors are designed and manufactured to perform only the functions necessary for a specific device or feature. Custom semiconductors are broadly classified according to their design method into Full Custom ICs, which design and manufacture circuits from scratch to meet user requirements, and Semi-Custom ICs, which design and manufacture circuits using parts of a standardized design.

[0114] Application-specific semiconductors are primarily used in communication systems, high-performance computing systems, consumer electronics, automobiles, industrial automation, medical devices, the military, and the aerospace industry; recently, they are being applied to AI semiconductors that execute the large-scale computations required for AI implementation with high performance and power efficiency.

[0115] Application-specific semiconductors (ASICs) are used as core components in communication systems, such as network routers, switches, and modems, performing data packet processing, protocol conversion, and signal processing to provide high throughput and low latency. In high-performance computing systems, ASICs serve as key components for high-speed and parallel processing, while in consumer electronics—including digital cameras, smartphones, tablets, and game consoles—ASICs provide high-performance and low-power solutions required to perform specific functions. In the automotive industry, ASICs are used to control various electronic systems within vehicles, and in industrial automation systems, they provide solutions for high-precision control and high-performance processing.

[0116] The application-specific integrated circuit to which the embodiment of the present invention is applied includes a memory in which an individual memory interface (I / F) is implemented, and may include a plurality of function blocks that request memory access. Each function block may be a Direct Memory Access (DMA) function block, a processor, a video processor, a cache controller, a decompression block, or a data path block. The basic configuration of the application-specific integrated circuit may include a transistor that amplifies or switches an electrical signal, a logic gate which is a circuit that performs a logical function by combining transistors, a memory cell that stores data, an analog circuit which is a circuit that processes continuous voltage or current by combining transistors, and an Intellectual Property Core (IP Core) such as a microprocessor, DSP, or graphics core that is pre-designed to perform a specific function.

[0117] An application-specific integrated circuit (or application-specific integrated circuit, ASIC) may include an individual memory I / F that interfaces with individual memory and an embedded memory I / F that interfaces with embedded memory. The individual memory I / F is connected to each function block to receive memory access signals (e.g., control signals, address signals, and data signals) and, based on these input signals, can generate signals to control the individual memory. The embedded memory I / F is connected to each function block to receive memory access signals (e.g., control signals, address signals, and data signals) and, based on these input signals, can generate modified memory access signals to control the embedded memory. The individual memory I / F and the embedded memory I / F are designed within the memory control block of the ASIC to provide a memory control structure that can be flexibly applied to both the individual memory and the embedded memory.

[0118] Additionally, an ASIC for an artificial neural network is composed of multiple neurons arranged in an array and multiple synapse circuits, each neuron is composed of a register, a microprocessor, and at least one input, and each synapse circuit may be configured to include memory for storing synapse weights. Here, each neuron of the ASIC may be connected to at least one other neuron through one of the multiple synapse circuits.

[0119] Although the present disclosure has been described as generally being implementable by a computing device, a person skilled in the art will be well aware that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that can be executed on one or more computers and / or as a combination of hardware and software.

[0120] Those skilled in the art of the present disclosure will understand that information and signals may be represented using any various different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0121] Those skilled in the art will understand that the various exemplary logic blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented by electronic hardware, various forms of programs or design code (referred to herein as software for convenience), or a combination of all such. To clearly illustrate this interoperability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in relation to their functions. Whether such functions are implemented as hardware or software depends on the design constraints imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementation decisions should not be interpreted as being outside the scope of this disclosure.

[0122] The various embodiments presented herein may be implemented as methods, devices, or manufactured articles using standard programming and / or engineering techniques. The term manufactured article includes a computer program, carrier, or medium accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical discs (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Additionally, the various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0123] It should be understood that the specific order or hierarchy of steps in the presented processes is merely an example of exemplary approaches. It should be understood that, based on design priorities, the specific order or hierarchy of steps in the processes may be rearranged within the scope of this disclosure. The appended method claims provide components of various steps in a sample order, but do not imply being limited to the specific order or hierarchy presented.

[0124] FIG. 4 illustrates an example of a block diagram of a computing system device (400) of the present disclosure.

[0125] Various operations of the system and method of the present disclosure may be performed by any suitable means capable of performing corresponding functions. Such means may include various hardware, software components, modules, and combinations thereof, including but not limited to circuits, processors, ASICs, and field programmable gate arrays (FPGAs).

[0126] Referring to FIG. 4, a computing system device (400) implementing an artificial neural network of the present disclosure may include a transceiver (410), memory (420), a database (430), and a processor (440). However, not all components shown in FIG. 4 are essential components of the computing system device (400). The computing system device (400) may be implemented with more components than those shown in FIG. 4, or with fewer components than those shown in FIG. 4. Furthermore, the transceiver (410), memory (420), and processor (440) may be implemented in the form of a single chip.

[0127] In one embodiment, the transceiver (410) can communicate with a terminal or other electronic device connected to the computing system device (400) via wired or wireless connection.

[0128] Various types of data, such as programs and files, such as applications, can be installed and stored in the memory (420). The processor (440) may access and use the data stored in the memory (420) or store new data in the memory (420). Additionally, one or more instructions may be stored in the memory (420). The processor (440) may execute one or more instructions stored in the memory.

[0129] The processor (440) controls the overall operation of the computing system device (400). Here, the processor (440) may be composed of at least one of a central processing unit (CPU), a graphics processing unit (GPU), an application integrated circuit, a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array, controllers, microcontrollers, microprocessors, and / or electrical units for performing other functions, or a plurality of electrically connected processors.

[0130] The processor (440) can control other components included in the computing system device (400) to perform operations for operating the computing system device (400).

[0131] The database (430) may store various training data for training a learning model. Additionally, the database (430) may store protein amino acid sequence information, protein structure information, simulation result information, etc., and in various embodiments, output data produced by the learning model may be stored. Although FIG. 4 is illustrated as including the database (430) in the computing system device (400), the database (430) may be provided outside the device. In this case, the database (430) may be connected to the computing system device (400) via a wired or wireless connection.

[0132] Additionally, the learning model of the present disclosure may be implemented outside the computing system device (400) (e.g., implemented on a cloud basis) or may be included inside the computing system device (400).

[0133] One embodiment of the present disclosure may also be implemented in the form of a recording medium comprising computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include both computer storage media and communication media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data. A communication medium typically includes computer-readable instructions, data structures, or program modules and includes any information transmission medium.

[0134] Throughout the entire disclosure, the device may include, but is not limited to, a server, smartphone, tablet PC, PC, TV, smart TV, mobile phone, PDA (personal digital assistant), speaker, laptop, media player, micro server, e-book object recognition device, digital broadcasting object recognition device, kiosk, MP3 player, digital camera, robot vacuum cleaner, home appliance, other mobile or non-mobile computing device, a watch, glasses, hair band and ring, etc. equipped with communication functions and data processing functions.

[0135] 6. Applications of Protein Structure Prediction Systems and Methods

[0136] The protein structure prediction system and method of the present disclosure can be applied to predict the complex structures and interactions of various biomolecules, such as proteins, DNA, RNA, and small molecules, and can be usefully utilized in the medical, pharmaceutical, and life science industries.

[0137] Specifically, the protein structure prediction system and method of the present disclosure can be used in new drug development and drug design. By accurately predicting how a new drug candidate binds to and affects a specific protein, the protein structure prediction system and method of the present disclosure can rapidly identify effective new drug substances and reduce unnecessary experiments, thereby significantly reducing development time and costs. In addition, it is utilized to reduce potential side effects by predicting in advance the likelihood of a drug reacting with other unintended molecules. This contributes to enhancing drug safety beyond simply increasing drug efficacy.

[0138] The protein structure prediction system and method of the present disclosure can be usefully applied in the fields of disease research and precision medicine. For example, by predicting how gene mutations cause changes in the structure and function of proteins, it can be used to elucidate the fundamental mechanisms of genetic diseases whose causes were previously difficult to identify. This can lay the foundation for precision medicine, which designs personalized treatments tailored to the characteristics of individual patients. Furthermore, by accurately predicting the binding structures of immune-related molecules, such as antigen-antibody complexes, during the development of immunotherapies, it can contribute to the development of new vaccines or treatments.

[0139] The protein structure prediction system and method of the present disclosure can be usefully applied in industrial fields related to protein engineering and enzymes, and can be utilized to design new proteins with specific functions or to optimize the functions of existing proteins. For example, it can play an important role in creating high-efficiency enzymes required for industrial processes or in developing disease-resistant crops by analyzing the interactions between plants and pathogens in the agricultural field.

[0140] The foregoing description of the present disclosure is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0141] The scope of the present disclosure is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present disclosure.

Claims

1. In a protein structure prediction system, Memory for storing one or more instructions; and At least one that executes the one or more instructions stored in the memory. Includes processors, The operation performed by the above one or more instructions is A step of converting protein data containing one or more protein amino acid sequence information into an embedding, The first model and the second model each generate a logit from the embedding, A step of generating a prediction distribution by applying a softmax function using temperature parameters to the logits generated by the first model and the second model, respectively. A step of using a prediction distribution generated through the first model as a soft target and obtaining a soft loss function by comparing it with a prediction distribution generated through the second model, and A protein structure prediction system comprising the step of obtaining a loss function using the soft loss function and performing learning to reduce the loss function.

2. A protein structure prediction system according to claim 1, wherein the first model performs protein structure prediction using a diffusion model.

3. A protein structure prediction system according to claim 1, wherein the second model is a stochastic circuit model.

4. A protein structure prediction system in which the probabilistic circuit model of paragraph 3 is a sum-product network or a probabilistic sentence decision diagram.

5. A protein structure prediction system according to claim 1, wherein the loss function is obtained using a soft loss function and a hard loss function, wherein the hard loss function is the difference obtained from the prediction distribution generated through a second model with the correct label as the hard target.

6. A protein structure prediction system according to claim 1, wherein, in the step of performing learning to reduce the loss function, learning is performed by additionally using one method selected from expectation maximization or stochastic gradient descent.

7. A protein structure prediction system according to claim 1, which calculates a result value for predicting a protein structure using a second model that has completed training, and calculates a result value for one or more selected from a group consisting of confidence, probability distribution, and uncertainty for the prediction.

8. A protein structure prediction method performed by at least one processor, A step of converting protein data containing one or more protein amino acid sequence information into an embedding, The first model and the second model each generate a logit from the embedding, A step of generating a prediction distribution by applying a softmax function using temperature parameters to the logits generated by the first model and the second model, respectively. A step of using a prediction distribution generated through the first model as a soft target and obtaining a soft loss function by comparing it with a prediction distribution generated through the second model, and A protein structure prediction method comprising the step of obtaining a loss function using the soft loss function and performing learning to reduce the loss function.

9. A protein structure prediction method according to claim 8, wherein the first model performs protein structure prediction using a diffusion model.

10. A protein structure prediction method according to claim 8, wherein the second model is a stochastic circuit model.

11. A protein structure prediction method according to claim 10, wherein the above-mentioned stochastic circuit model is a sum-product network or a stochastic sentence decision diagram.

12. A protein structure prediction method according to claim 8, wherein the above loss function is calculated by additionally using a loss function obtained by calculating the difference between the prediction of the second model and the actual correct label.

13. A protein structure prediction method according to claim 8, wherein, in the step of performing learning to reduce the loss function, one method selected from expectation maximization or stochastic gradient descent is additionally used to perform learning.

14. A protein structure prediction method according to claim 8, wherein a result value for predicting a protein structure is calculated using a second model that has completed training, and a result value for one or more selected from a group consisting of confidence, probability distribution, and uncertainty for the prediction.

15. A program stored on a computer-readable recording medium to execute the method of any one of paragraphs 8 through 14 on a computer.

Images