Method for learning 3D geometry of molecule and target physical property prediction method including same

Abstract

A method for learning the 3D geometry of a molecule and a target physical property prediction method including same according to an embodiment of the present invention are methods by which a computing system including a memory and a processor learns the 3D geometry of a molecule and predicts a target physical property. The method for learning the 3D geometry of a molecule includes the steps of: performing first pre-training based on denoising on the basis of a three-dimensional conformer encoder that uses, as input, three-dimensional molecular data which is data specifying the molecular structure at the three-dimensional level; performing second pre-training based on knowledge distillation on the basis of the first pre-trained three-dimensional conformer encoder and a two-dimensional graph encoder (2D graph encoder) that uses, as input, two-dimensional molecular data which is data specifying the molecular structure at the two-dimensional level; performing third pre-training based on fine-tuning on the basis of the second pre-trained two-dimensional graph encoder; and providing the third pre-trained two-dimensional graph encoder.

Description

A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same

[0001] The present invention relates to a method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same. More specifically, the present invention relates to a method for learning the 3D geometric structure of a molecule by pre-training an encoder based on 2D data based on the D&D (Denoise and Distill) methodology that applies 3D data, and a method for predicting target properties including the same.

[0002] Molecular property prediction plays a crucial role in various fields of chemistry and life sciences, such as drug development and / or new material design. However, obtaining the accurate labeled data required for high-precision predictions is extremely expensive. Therefore, in many cases, learning effective molecular representations from large-scale, unlabeled molecular data is essential.

[0003] Traditionally, two-dimensional graph-based molecular dictionary learning techniques have been widely used. These methods represent molecules as graphs of atoms and bonds and use graph neural networks to predict molecular properties. However, these two-dimensional graph-based approaches can lose their graph topology during data augmentation, limiting their ability to significantly improve prediction performance.

[0004] This has led to the recent emergence of pre-training methods using 3D conformers. 3D conformers provide information on the positions of atoms in physical space, and this information can be used to more accurately learn the chemical properties of molecules. In particular, denoising 3D structures has proven to be highly effective in learning the force fields generated during the stabilization process.

[0005] However, 3D conformer-based learning methods often require accurate 3D structural information for downstream tasks, making them computationally expensive for application to large-scale data sets. For example, generating 3D conformers for new molecules requires expensive quantum mechanical calculations.

[0006] Therefore, to overcome the above limitations, an efficient prediction model that does not require 3D information is required.

[0007] One embodiment of the present invention has been devised to solve the problems of the prior art as described above, and its purpose is to provide a method for learning a 3D geometric structure of a molecule by pre-training an encoder based on 2D data based on a D&D (Denoise and Distill) framework that applies 3D data, and a method for predicting target properties including the same.

[0008] Specifically, one embodiment of the present invention provides a method for learning a 3D geometric structure of a molecule and a method for predicting target properties including the same, which performs denoising-based learning through a 3D conformer encoder, performs learning to transfer (distillate) the knowledge learned in the 3D conformer encoder to a 2D graph encoder, and performs learning to optimize (fine-tuning) the 2D graph encoder that has received the knowledge to a specific property.

[0009] However, the technical problems to be solved by the present invention and embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

[0010] A method for learning a 3D geometric structure of a molecule and a method for predicting a target property including the same according to one embodiment of the present invention are methods in which a computing system including a memory and a processor learns a 3D geometric structure of a molecule and predicts a target property, the method comprising: a step of performing a first pre-training based on denoising based on a 3D conformer encoder that inputs 3D molecular data, which is data specifying a 3D level molecular structure; a step of performing a second pre-training based on knowledge distillation based on the first pre-trained 3D conformer encoder and a 2D graph encoder that inputs 2D molecular data, which is data specifying a 2D level molecular structure; a step of performing a third pre-training based on fine-tuning based on the second pre-trained 2D graph encoder; and a step of providing the third pre-trained 2D graph encoder.

[0011] In another aspect, the step of performing the first pre-learning includes the step of inserting predetermined noise into the 3D molecular data, the step of restoring the 3D molecular data with the noise inserted into it to the original 3D molecular data, and the step of training the 3D conformer encoder to learn the molecular structure at the 3D level.

[0012] In another aspect, the step of performing the first pre-learning further includes the step of performing data representation learning independent of rotation and translation in three-dimensional space based on a predetermined SE(3) transformation invariant architecture.

[0013] In another aspect, the step of performing the second pre-learning includes the step of performing distillation learning using the 3D Conformer Denoising Encoder, which is the first pre-learned 3D conformer encoder, as a teacher model and the 2D graph encoder as a student model.

[0014] In another aspect, the step of performing the transfer learning includes the step of training the two-dimensional graph encoder so that the output representation of the two-dimensional graph encoder follows the output representation of the three-dimensional conformer denoising encoder.

[0015] In another aspect, the step of performing the transfer learning further includes the step of performing graph-level knowledge transfer (D&D-GRAPH) to minimize the difference between the graph-level representation output by the two-dimensional graph encoder and the graph-level representation output by the three-dimensional conformer denoising encoder.

[0016] In another aspect, the step of performing the transfer learning further includes the step of performing node-level knowledge transfer (D&D-NODE) ​​that minimizes the difference between the node-level representation output by the two-dimensional graph encoder and the node-level representation output by the three-dimensional conformer denoising encoder.

[0017] In another aspect, the step of performing the transfer learning further includes the step of freezing at least some parameters of the 3D conformer denoising encoder.

[0018] In another aspect, the step of performing the third pre-training includes the step of performing a downstream task of optimizing the second pre-trained two-dimensional graph encoder, which is a two-dimensional graph transition encoder, to a predetermined property.

[0019] In another aspect, the step of providing the third pre-trained two-dimensional graph encoder includes the step of applying the two-dimensional graph fine-tuning encoder, which is the third pre-trained two-dimensional graph encoder, to a predetermined multitasking model.

[0020] Meanwhile, a system for learning a 3D geometric structure of a molecule and predicting target properties according to one embodiment of the present invention comprises at least one memory; And at least one processor for learning a 3D geometric structure of a molecule and predicting a target property by reading out at least one application stored in the memory; wherein the instructions of the processor include instructions for performing a first pre-learning based on denoising based on a 3D conformer encoder that inputs 3D molecular data, which is data specifying a 3D level molecular structure, a step of performing a second pre-learning based on knowledge distillation based on the first pre-learned 3D conformer encoder and a 2D graph encoder that inputs 2D molecular data, which is data specifying a 2D level molecular structure, a step of performing a third pre-learning based on fine-tuning based on the second pre-learned 2D graph encoder, and a step of providing the third pre-learned 2D graph encoder.

[0021] A method for learning a 3D geometric structure of a molecule and a method for predicting target properties including the same according to one embodiment of the present invention have the effect of enabling a 2D level encoder to efficiently include 3D level information by pre-training a 2D data-based encoder based on a D&D (Denoise and Distill) framework that applies 3D data.

[0022] Accordingly, the method for learning the 3D geometric structure of a molecule and the method for predicting target properties including the same according to one embodiment of the present invention have the effect of providing a high-performance encoder that performs molecular property prediction based on 3D level information through 2D data (e.g., 2D molecular graph, etc.).

[0023] Thus, the method for learning the 3D geometric structure of a molecule according to one embodiment of the present invention and the method for predicting target properties including the same have the effect of significantly reducing the computational cost while achieving high prediction accuracy by utilizing 3D level information.

[0024] In addition, a method for learning a 3D geometric structure of a molecule according to an embodiment of the present invention and a method for predicting target properties including the same perform denoising-based learning through a 3D conformer encoder and then perform learning to transfer (distillate) the knowledge learned in the 3D conformer encoder to a 2D graph encoder, thereby providing a high-performance encoder that combines the strengths of 3D-based molecular representation learning and the efficiency of 2D-based molecular representation learning.

[0025] In addition, according to one embodiment of the present invention, a method for learning a 3D geometric structure of a molecule and a method for predicting target properties including the same implement a two-dimensional level encoder that effectively learns generalized knowledge from a given data set, thereby having the effect of maintaining high prediction performance even in an environment where label data is limited.

[0026] In addition, the method for learning the 3D geometric structure of a molecule according to one embodiment of the present invention and the method for predicting target properties including the same have the effect of performing learning to optimize (fine-tune) a 2D graph encoder that has received knowledge for a specific property, thereby performing accurate and generalized prediction suitable for a specific application case.

[0027] In addition, the method for learning the 3D geometric structure of a molecule according to one embodiment of the present invention and the method for predicting target properties including the same have the effect of improving the efficiency of learning and prediction for large-scale data while simultaneously increasing its versatility and expandability by providing the D&D (Denoise and Distill) framework as described above.

[0028] That is, the method for learning the 3D geometric structure of a molecule according to one embodiment of the present invention and the method for predicting target properties including the same provide a generalized learning framework that can be applied to various molecular science problems, not limited to specific molecular property prediction problems, thereby providing an improved encoder that can be widely utilized in various research and industrial applications.

[0029] However, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood from the description below.

[0030] FIG. 1 illustrates an example block diagram of a computing system implementing a model encoder training service according to one embodiment of the present invention.

[0031] FIG. 2 illustrates an example block diagram of a computing device implementing a model encoder training service according to one embodiment of the present invention.

[0032] FIG. 3 illustrates an example block diagram from another aspect for a computing device implementing a model encoder training service according to one embodiment of the present invention.

[0033] Figures 4 and 5 illustrate examples of conceptual diagrams for explaining a multitasking learning model according to one embodiment of the present invention.

[0034] Figure 6 illustrates an internal block diagram of a multitasking learning model according to one embodiment of the present invention.

[0035] FIG. 7 illustrates an example of a conceptual diagram for explaining a multi-tasking model learning method according to one embodiment of the present invention.

[0036] FIG. 8 illustrates a block flow diagram for explaining a multi-tasking model learning method according to one embodiment of the present invention.

[0037] FIG. 9 illustrates a block flow diagram for explaining a multi-tasking learning model training method according to one embodiment of the present invention.

[0038] FIG. 10 illustrates an example of a conceptual diagram for explaining a multi-tasking learning model training method according to one embodiment of the present invention.

[0039] FIG. 11 and FIG. 12 illustrate examples of drawings for explaining a method for calculating regression loss according to one embodiment of the present invention.

[0040] FIG. 13 illustrates an example of a drawing for explaining an integrated latent space mapping method according to one embodiment of the present invention.

[0041] FIG. 14 and FIG. 15 illustrate examples of drawings for explaining a consistency loss calculation method according to one embodiment of the present invention.

[0042] FIG. 16 and FIG. 17 illustrate examples of drawings for explaining a mapping loss calculation method according to one embodiment of the present invention.

[0043] FIG. 18 illustrates an example of a drawing for explaining an integrated loss calculation method according to one embodiment of the present invention.

[0044] FIG. 19 illustrates a block flow diagram for explaining a method for learning a 3D geometric structure of a molecule and a method for predicting target properties including the same according to one embodiment of the present invention.

[0045] Figure 20 illustrates an example of a conceptual diagram for explaining the first pre-learning according to one embodiment of the present invention.

[0046] Figure 21 illustrates an example of a conceptual diagram for explaining second pre-learning according to one embodiment of the present invention.

[0047] FIG. 22 illustrates an example of a drawing for explaining a method for determining a model parameter freezing range during second pre-learning according to one embodiment of the present invention.

[0048] Figure 23 illustrates an example of a conceptual diagram for explaining the third pre-learning according to one embodiment of the present invention.

[0049] The present invention is capable of various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, as well as the methods for achieving them, will become clear with reference to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms. In the following embodiments, the terms "first," "second," etc. are not used in a limiting sense but are used for the purpose of distinguishing one component from another. Furthermore, the singular expression includes the plural expression unless the context clearly indicates otherwise. Furthermore, terms such as "include" or "have" indicate the presence of a feature or component described in the specification, and do not preemptively exclude the possibility that one or more other features or components may be added. Furthermore, in the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to what is shown.

[0050] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same drawing reference numerals, and redundant descriptions thereof will be omitted.

[0051]

[0052] [An example system implementing a model encoder training service]

[0053] Below, an exemplary system providing a model encoder training service that pre-trains a two-dimensional data-based encoder based on the D&D (Denoise and Distill) methodology that applies three-dimensional data is described in detail with reference to the attached drawings.

[0054] FIG. 1 illustrates an example block diagram of a computing system implementing a model encoder training service according to one embodiment of the present invention.

[0055] Referring to FIG. 1, a computing system (1000) implementing a model encoder training service of the present invention includes a user computing device (110), a server computing system (130), and a training computing system (150), and the devices are capable of communicating via a network (170).

[0056] A multitasking model learning method according to one embodiment of the present invention and a multitasking performance method using a machine learning model learned based thereon may be implemented and provided locally by a user computing device (110), implemented and provided locally by a server computing system (130) communicating with the user computing device (110) in the form of a web service, or implemented and provided by a user computing device (110) and a server computing system (130) in connection with each other.

[0057] At this time, in the embodiment, the user computing device (110) and / or the server computing system (130) can train the machine learning model (120 and / or 140) through interaction with a training computing system (150) communicatively connected via a network (170). The training computing system (150) may be separate from the server computing system (130) or may be part of the server computing system (130).

[0058] And at this time, the artificial intelligence model can be 1) trained directly locally by the user computing device (110), 2) trained by the server computing system (130) and the user computing device (110) interacting with each other through a network (170), and 3) trained by a separate training computing system (150) using various training techniques and learning techniques. And the artificial intelligence model trained by the training computing system (150) can be implemented in a manner of being provided / updated by transmitting the model to the user computing device (110) and / or the server computing system (130) through a network (170).

[0059] In some embodiments, the training computing system (150) may be part of a server computing system (130) or part of a user computing device (110).

[0060] The user computing device (110) may include any type of computing device, such as a smart phone, a mobile phone, a digital broadcasting device, a personal digital assistant (PDA), a portable multimedia player (PMP), a desktop, a wearable device, an embedded computing device, and / or a tablet PC.

[0061] Such a user computing device (110) includes at least one processor (111) and memory (112). Here, the processor (111) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0062] The memory (112) may include one or more non-transitory / transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof, and may include web storage of a server that performs a memory storage function on the Internet. The memory (112) may store data (113) and commands (114) necessary for the at least one processor (111) to perform functional operations, such as training an artificial intelligence model or executing multi-tasking learning through an artificial intelligence model.

[0063] In one embodiment, the user computing device (110) may store at least one machine learning model (120).

[0064] In detail, the machine learning model (120) may be various machine learning models such as multiple neural networks (e.g., deep neural networks) or other types of machine learning models including nonlinear models and / or linear models, and may be composed of a combination thereof.

[0065] At this time, the neural network may include at least one of feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, and / or other types of neural networks.

[0066] In one embodiment, the user computing device (110) may receive at least one machine learning model (120) from a server computing system (130) via a network (170), store the model in a memory (112), and then execute the stored machine learning model (120) by a processor (111) to perform multi-tasking learning, etc.

[0067] In another embodiment, the server computing system (130) may include at least one machine learning model (140), perform operations through the machine learning model (140), and communicate data related thereto with the user computing device (110) to provide a model encoder training service to the user.

[0068] For example, a user computing device (110) can perform a model encoder training service in a manner that a server computing system (130) provides output for a user's input using a machine learning model (140) via the web.

[0069] Additionally, the artificial intelligence model may be implemented in such a way that at least some of the machine learning models (120 and / or 140) are executed on a user computing device (110) and the rest are executed on a server computing system (130).

[0070] In addition, the user computing device (110) may include at least one input component (121) that detects a user's input. For example, the user input component (121) may include a touch sensor (e.g., a touch screen and / or a touch pad, etc.) that detects a touch of a user's input medium (e.g., a finger or a stylus), an image sensor that detects a user's motion input, a microphone, a button, a mouse, and / or a keyboard that detects a user's voice input, etc. In addition, the user input component (121) may include an interface and an external controller when receiving an input to an external controller (e.g., a mouse and / or a keyboard, etc.) through an interface.

[0071] The server computing system (130) includes at least one processor (131) and memory (132). Here, the processor (131) may be composed of at least one or a plurality of processors electrically connected among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0072] And the memory (132) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. This memory (132) may store data (133) and instructions (134) necessary for the processor (131) to perform functional operations such as training an artificial intelligence model or executing multi-tasking learning through an artificial intelligence model.

[0073] In one embodiment, the server computing system (130) may be implemented to include at least one computing device. For example, the server computing system (130) may be implemented to operate multiple computing devices according to a sequential computing architecture, a parallel computing architecture, or a combination thereof. Additionally, the server computing system (130) may include multiple computing devices connected via a network (170).

[0074] Additionally, the server computing system (130) may store at least one machine learning model (140). For example, the server computing system (130) may include a neural network and / or other multi-layer nonlinear model as the machine learning model (140). Exemplary neural networks may include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0075] The training computing system (150) includes at least one processor (151) and memory (152). Here, the processor (151) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0076] And the memory (152) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. This memory (152) may store data (153) and instructions (154) necessary for the processor (151) to perform learning of an artificial intelligence model, etc.

[0077] For example, the training computing system (150) may include a model trainer (160) that trains a machine learning model (120 and / or 140) stored in the user computing device (110) and / or server computing system (130) using various training or learning techniques, such as backpropagation of errors (according to the framework illustrated in FIG. 3).

[0078] For example, such a model trainer (160) may perform updates to one or more parameters of a machine learning model (120 and / or 140) in a backpropagation manner based on a defined loss function.

[0079] In some implementations, performing backward propagation of errors may include performing truncated backpropagation through time. The model trainer (160) may perform a number of generalization techniques (e.g., weight reduction, dropout, and / or knowledge distillation) to improve the generalization ability of the trained machine learning model (120 and / or 140).

[0080] In particular, the model trainer (160) can train the machine learning model (120 and / or 140) based on a series of training data (161). Here, the training data (161) can include data in different formats, such as, for example, images, audio samples, and / or text. Examples of image types that can be used can include video frames, LiDAR point clouds, X-ray images, computed tomography scans, hyperspectral images, and / or various other forms of images.

[0081] Such training data (161) may be provided by a user computing device (110) and / or a server computing system (130). When the training computing device trains a machine learning model (120 and / or 140) on specific data of the user computing device (110), the machine learning model (120 and / or 140) may be characterized as a personalized model.

[0082] And the model trainer (160) includes computer logic utilized to provide the desired function.

[0083] Additionally, the model trainer (160) may be implemented as hardware, firmware, and / or software that controls a general-purpose processor. In one implementation, the model trainer (160) includes a program file stored in a storage device, which may be loaded into a memory (152) and executed by one or more processors (151). In another implementation, the model trainer (160) includes one or more sets of computer-executable data (153) and instructions (154) stored in a tangible computer-readable storage medium, such as a RAM hard disk or an optical or magnetic medium.

[0084] The network (170) includes, but is not limited to, a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), a Wireless Local Area Network (Wireless LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, and / or a Digital Multimedia Broadcasting (DMB) network.

[0085] In general, communication over a network (170) may be performed using any type of wired and / or wireless connection, using various communication protocols (e.g., TCP / IP, HTTP, SMTP, and / or FTP), encodings or formats (e.g., HTML and / or XML), and / or protection schemes (e.g., VPN, Secure HTTP, and / or SSL).

[0086] FIG. 2 illustrates an example block diagram of a computing device implementing a model encoder training service according to one embodiment of the present invention.

[0087] As shown in FIG. 2, the computing device (100) included in the user computing device (110), the server computing system (130), and the training computing system (150) includes a plurality of applications (e.g., Application 1 to Application N). Each application may include a machine learning library and one or more machine learning models. For example, the applications may include an image processing (e.g., Detection, Classification, and / or Segmentation, etc.) application, a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and / or a chat-bot application.

[0088] In an embodiment, the computing device (100) may include a model trainer (160) for training an artificial intelligence model, and may store and operate the trained artificial intelligence model to provide output data according to predetermined input data (for example, material characteristic information and / or material property information, etc.).

[0089] Each application of the computing device (100) may communicate with a number of other components of the computing device (100), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In one embodiment, each application may communicate with each device component using an API (e.g., a public API). In one embodiment, the API used by each application may be specific to that application.

[0090] FIG. 3 illustrates an example block diagram from another aspect for a computing device implementing a model encoder training service according to one embodiment of the present invention.

[0091] Referring to FIG. 3, a computing device (200) includes a plurality of applications (e.g., Application 1 to Application N). Each application may communicate with a central intelligence layer. For example, the applications may include an image processing application, a text messaging application, an email application, a dictation application, a virtual keyboard application, and / or a browser application. In one embodiment, each application may communicate with the central intelligence layer (and models stored therein) using an API (e.g., a common API across all applications).

[0092] The central intelligence layer may include multiple machine learning models. For example, as illustrated in FIG. 3, at least some of the machine learning models may be provided to each application and managed by the central intelligence layer. In other implementations, two or more applications may share a single machine learning model. For example, in some implementations, the central intelligence layer may provide a single model to all applications. In some implementations, the central intelligence layer may be included within the operating system of the computing device (200) or implemented differently.

[0093] The central intelligence layer may communicate with a central device data layer. The central device data layer may be a centralized data repository for the computing device (200). As illustrated in FIG. 3, the central device data layer may communicate with a number of other components of the computing device (200), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer may communicate with each device component using an API (e.g., a private API).

[0094] The techniques described herein may refer to servers, databases, software applications, and other computer-based systems, as well as actions taken and information transmitted to or from such systems. It will be appreciated that the inherent flexibility of computer-based systems allows for a wide range of possible configurations, combinations, and division of labor and functionality between and among components. For example, the processes described herein may be implemented using a single device or component, or multiple devices or components operating in combination. Databases and applications may be implemented on a single system or in a distributed system across multiple systems. Distributed components may operate sequentially or in parallel.

[0095]

[0096] Multitasking Learning Model (MtLM)

[0097] FIGS. 4 and 5 illustrate examples of conceptual diagrams for explaining a multi-task learning model (MtLM) according to one embodiment of the present invention.

[0098] Referring to FIGS. 4 and 5, a multi-task learning model (MtLM) (Geometrically Aligned Transfer Encoder Model) according to an embodiment of the present invention may be a machine learning model that aligns fragmented knowledge data (for example, latent vectors, etc.) in each task's latent space through geometric transfer in a single integrated latent space (M: Manifold) in order to process multi-tasks for an integrated output satisfying multiple domains.

[0099] In other words, the multi-tasking learning model (MtLM) according to the embodiment not only simultaneously learns knowledge data according to various domains but also efficiently learns relationships between multiple domains, thereby expanding the learning area and simultaneously performing effective multi-tasking learning that implements batch learning of local patterns according to each domain and common principles between multiple domains.

[0100] Accordingly, the multi-tasking learning model (MtLM) can directly improve the processing performance and accuracy of various multi-tasking tasks based on the model learned as above.

[0101] In the embodiment, a multi-tasking learning model (MtLM) will be described as an example of a relationship between a material and multiple properties, and a learning model that multi-tasks multiple tasks, including a first task of predicting the characteristics of a first property for a material, a second task of predicting the characteristics of a second property for a material, etc. However, it should be understood that the present invention is not limited to a learning method and a prediction method for multi-tasking the relationship between a material and multiple properties, and can be applied to all kinds of tasks that require performing multiple tasks simultaneously, such as the relationship between a material and multiple properties.

[0102] In an embodiment, such a multi-task learning model (MtLM) can perform pre-training based on certain experimental data.

[0103] Here, the experimental data according to the embodiment may be data including predetermined material-specific characteristic information and material property-specific information as learning data used for training a multi-task learning model (MtLM).

[0104] At this time, the material characteristic information according to the embodiment may be information that specifies the unique characteristics possessed by a given material.

[0105] For example, material characteristic information may include at least one of a given material name, molecular structure, and / or chemical formula.

[0106] In addition, the material property specific information according to the embodiment may be information that specifies the data value that a given material has for a given material property.

[0107] For example, material property specific information may include property (i.e., domain) values ​​such as boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity of a given material.

[0108] Meanwhile, the multi-task learning model (MtLM) that performed pre-learning as described above in the embodiment can input predetermined material-specific characteristic information and / or material property-specific information, and output predicted data based on the input information and learned knowledge.

[0109] As an example, a multi-task learning model (MtLM) can input certain material-specific characteristic information and output predicted material property-specific information based on the input information and learned knowledge.

[0110] In another embodiment, a multi-task learning model (MtLM) can input certain material property information and output predicted material property information based on the input information and learned knowledge.

[0111] In another embodiment, the multi-task learning model (MtLM) may include a reverse-engineered model that receives certain material-specific characteristic information and material-specific property information as input, and outputs optimal material-specific characteristic information and material-specific property information predicted based on the input information and learned knowledge.

[0112] In the following examples, molecular structural formula data representing material-specific characteristic information is used as input data, and characteristic value data of each task's physical properties is used as output data.

[0113] FIG. 6 illustrates an internal block diagram of a multi-tasking learning model (MtLM) according to one embodiment of the present invention.

[0114] Referring to FIG. 6, in another aspect, a multi-task learning model (MtLM) according to an embodiment may include at least one embedding module (EBM: Embedding Module), an encoder module (ECM: Encoder Module), a regressor module (RGM: Regressor Module), a transfer module (TFM: Transfer Module), an inverse transfer module (ITM: Inverse Transfer Module), a perturbation module (PBM: Perturbation Module), and a loss calculation module (LCM: Loss Calculation Module).

[0115] In detail, the embedding module (EBM) according to an embodiment of the present invention may be a pre-encoder module that converts a predetermined input data into an embedding vector.

[0116] Specifically, the embedding module compresses high-dimensional data, such as molecular structure data, into a low-dimensional representation, such as an embedding vector, thereby reducing the dimensionality of the input to be processed by the encoder, thereby improving computational efficiency and learning speed, and allows the encoder to focus on and learn important features in the molecular structure for the pre-learning task.

[0117] Through this, useful features from the model learned for the source task can be easily applied to the model that learns the target task, and generalization is possible for overlapping features between different domains, allowing for effective transfer learning for new domains / tasks.

[0118] In other words, the embedding module (EBM) may be a module that converts specific input data into a vector format by projecting it into a predetermined embedding space.

[0119] As an example, a graph neural network (GNN) suitable for extracting molecular structural features can be used as an embedding module (EBM), and for example, an embedding vector for input data can be provided based on a DMPNN (Directed Message Passing Neural Network) structure.

[0120] In addition, the encoder module (ECM) according to an embodiment of the present invention may be a module that takes a predetermined embedding vector as input and converts the input embedding vector into a latent vector by projecting it into a latent space corresponding to the task.

[0121] In other words, the encoder module (ECM) can be a module that extracts key features of the input embedding vector and expresses them in the corresponding latent space. In detail, the encoder module (ECM) can extract important features from the features of the embedding vector, perform data compression such as removing unnecessary information or noise, and compress the data into a low-dimensional space, and output a latent vector, which is a representation in the latent space.

[0122] In an embodiment, such an encoder module (ECM) may include a plurality of encoder modules (ECMs) corresponding to each of a plurality of domains.

[0123] In an embodiment, the encoder module (ECM) may include a first encoder module (ECM) corresponding to a first domain (e.g., boiling point) and a second encoder module (ECM) corresponding to a second domain (e.g., melting point).

[0124] In another embodiment, the encoder module (ECM) may include a third encoder module (ECM) for performing a first task of predicting solubility for a first solvent in response to a third domain (e.g., solubility), and a fourth encoder module (ECM) for performing a second task of predicting solubility for a second solvent. That is, another embodiment may include a case of multitasking for different tasks for the same domain. Of course, a multitasking model that integrates multitasking for different domains and multitasking for different tasks in the same domain may also be included in one embodiment of the present invention.

[0125] Below, we will explain based on the fact that different domains represent different tasks.

[0126] At this time, in the embodiment, one of the plurality of encoder modules (ECMs) may be a source encoder module (ECM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0127] In addition, any one of the remaining encoder modules (ECMs) excluding the source encoder module (ECM) may be a target encoder module (ECM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0128] In addition, the regressor module (RGM) according to an embodiment of the present invention may be a head module that takes a predetermined latent vector as input and generates a final prediction value according to the input latent vector. That is, in the embodiment, the regressor module is used as an example of the head module.

[0129] These regression modules (RGMs) can directly participate in generating the final output and thus determine the model's predictive performance.

[0130] Additionally, in the embodiment, the regressor module (RGM) may include a plurality of regressor modules (RGMs) corresponding to each of a plurality of domains.

[0131] In an embodiment, the regressor module (RGM) may include a first regressor module (RGM) corresponding to a first domain (e.g., boiling point) and a second regressor module (RGM) corresponding to a second domain (e.g., melting point).

[0132] At this time, in the embodiment, one of the plurality of regressor modules (RGMs) may be a source regressor module (RGM), which is a regressor module (RGM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0133] In addition, any one of the remaining regressor modules (RGMs) excluding the source regressor module (RGM) may be a target regressor module (RGM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0134] In addition, a transfer module (TFM) according to an embodiment of the present invention may be a module that converts a predetermined latent vector into a transfer vector by mapping it to a latent space of another task.

[0135] In detail, in an embodiment, the transfer module (TFM) can transform a specific latent vector into a transition vector by mapping it to the latent space of another task based on Riemannian geometry.

[0136] In this process, the transfer module (TFM) can implement geometric alignment between each mapped task according to an embodiment of the present invention. A detailed description of this will be provided later in the multitasking model learning method.

[0137] That is, in the embodiment, the transfer module (TFM) can effectively perform the transfer of knowledge data between multiple tasks by mapping the latent vector according to the first task to the latent space according to the second task through geometric alignment according to the embodiment of the present invention.

[0138] At this time, in the embodiment, the transfer module (TFM) can support data processing that improves the accuracy and consistency of the transformed vector (i.e., the transition vector) by utilizing an autoencoder structure.

[0139] Additionally, in the embodiment, the transfer module (TFM) may include a plurality of transfer modules (TFMs) corresponding to each of the plurality of domains.

[0140] In an embodiment, the transfer module (TFM) may include a first transfer module (TFM) corresponding to a first domain (e.g., boiling point) and a second transfer module (TFM) corresponding to a second domain (e.g., melting point).

[0141] At this time, in the embodiment, one of the plurality of transfer modules (TFMs) may be a source transfer module (TFM), which is a transfer module (TFM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0142] In addition, any one of the remaining transfer modules (TFMs) excluding the source transfer module (TFM) may be a target transfer module (TFM) that is a transfer module (TFM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0143] In addition, the inverse module (ITM) according to an embodiment of the present invention may be a module that reconstructs a transition vector mapped and transformed into a latent space of another task by a transfer module (TFM) so that it is mapped back to the original latent space.

[0144] Thus, in the embodiment, the inverse module (ITM) can generate a vector (hereinafter, an inverse vector) that is a reconstructed and transformed vector back to its original state.

[0145] At this time, the inverse module (ITM) in the embodiment can improve the stability of the above-described reconstruction process and the accuracy and consistency of the corresponding transition vector by utilizing the autoencoder structure.

[0146] In an embodiment, such an inverse module (ITM) may include a plurality of inverse modules (ITMs), each corresponding to a plurality of domains.

[0147] In an embodiment, the inverse module (ITM) may include a first inverse module (ITM) corresponding to a first domain (e.g., boiling point) and a second inverse module (ITM) corresponding to a second domain (e.g., melting point).

[0148] At this time, in the embodiment, one of the plurality of inverse modules (ITMs) may be a source inverse module (ITM), which is an inverse module (ITM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0149] In addition, any one of the remaining inverse modules (ITMs) excluding the source inverse module (ITM) may be a target inverse module (ITM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0150] In addition, a perturbation module (PBM) according to an embodiment of the present invention may be a module that generates a plurality of perturbation vectors by applying a predetermined change to a predetermined embedding vector.

[0151] In detail, in an embodiment, the perturbation module (PBM) may be a module that generates a plurality of perturbation vectors (i.e., perturbation points) around a specific embedding vector by applying a change that moves the specific embedding vector in a predetermined direction.

[0152] At this time, the plurality of generated perturbation vectors are designed to maintain a relative distance from the corresponding embedding vector, thereby effectively assisting geometric alignment.

[0153] That is, the above-mentioned perturbation module (PBM) can help align the coordinate systems between the source task and the target task by generating multiple perturbation vectors to assist in the geometric alignment of the model.

[0154] Additionally, in the embodiment, the perturbation module (PBM) can calculate the distance between a predetermined embedding vector and a plurality of perturbation vectors generated based on the embedding vector, and support matching the displacement between the source task and the target task based on the calculated distance.

[0155] This allows the perturbation module (PBM) to more easily maintain consistency in the latent space for the model.

[0156] According to an embodiment, the perturbation module (PBM) can prevent overfitting of the model and improve generalization performance by forcing a relationship between a given embedding vector and multiple perturbation vectors generated based on the embedding vector to be maintained.

[0157] In addition, the loss calculation module (LCM) according to an embodiment of the present invention may be a module that calculates various loss functions based on various vectors obtained through a multi-task learning model (MtLM).

[0158] In an embodiment, the loss calculation module (LCM) may calculate regression loss, autoencoder loss, consistency loss, mapping loss, distance loss, and / or integrated loss according to an embodiment of the present invention. A detailed description thereof will be provided later in the multi-task model learning method.

[0159] This allows the Loss Calculation Module (LCM) to support regularization and learning for different parts of the model, and to provide feedback for model learning, enabling model optimization.

[0160] Meanwhile, in an embodiment of the present invention, the multi-tasking learning model (MtLM) can perform model optimization and update through various data processing processes linked with the modules described above.

[0161] For example, the multi-task learning model (MtLM) can perform model optimization and parameter update in conjunction with the modules described above based on the AdamW optimization algorithm, etc.

[0162] In this way, in the embodiment of the present invention, the multi-tasking learning model (MtLM) not only simultaneously learns knowledge data according to various domains, but also efficiently learns relationships between multiple domains, thereby expanding the learning area and simultaneously performing effective multi-tasking learning that implements batch learning of local patterns according to each domain and common principles between multiple domains.

[0163] Accordingly, the multi-tasking learning model (MtLM) can directly improve the processing performance and accuracy of various multi-tasking tasks based on the model learned as above.

[0164]

[0165] [2D Graph Encoder (f_2D: 2D Graph Encoder)]

[0166] A two-dimensional graph encoder (f_2D) according to one embodiment of the present invention may be an encoder that takes two-dimensional structural data of a molecule (hereinafter, two-dimensional molecule data) as input.

[0167] This 2D graph encoder (f_2D) can serve as a student model that receives knowledge distillation from a 3D conformer encoder, which will be described later. In other words, the 2D graph encoder (f_2D) can be an entity that receives knowledge distillation from a 3D conformer encoder.

[0168] Accordingly, the 2D graph encoder (f_2D) can predict 3D characteristic information with high accuracy through given 2D molecular data (e.g., 2D molecular graph, etc.) even if the 3D structural data of the molecule (hereinafter, 3D molecular data) is insufficient.

[0169] Here, the three-dimensional characteristic information according to the embodiment may mean information specifying a molecular characteristic (for example, information on the behavioral aspect of a molecule, information on the physical force field of a molecule, etc.) provided through a molecular shape according to three-dimensional molecular data among various physical properties of a molecule (for example, information on the physical force field of a molecule, etc.).

[0170] In detail, in the embodiment, a two-dimensional graph encoder (f_2D) can learn a molecular structure according to two-dimensional molecular data while following a representation learned from a three-dimensional conformer encoder.

[0171] Thus, the 2D graph encoder (f_2D) can very accurately predict the 3D characteristic information described above through 2D molecular data even in the absence of 3D molecular data.

[0172] In other words, in the embodiment, the two-dimensional graph encoder (f_2D) can predict three-dimensional characteristic information with high accuracy based on given two-dimensional molecular data through a process of learning a representation similar to a three-dimensional conformer encoder.

[0173] Through this, the 2D graph encoder (f_2D) can significantly reduce the data processing cost required to obtain 3D characteristic information, while improving the model performance for performing various molecular / material property prediction tasks.

[0174] More specifically, in the embodiment, the above 2D graph encoder (f_2D) can obtain a 2D graph embedding.

[0175] In detail, in the embodiment, the two-dimensional graph encoder (f_2D) can receive two-dimensional molecular data (for example, a two-dimensional molecular graph expressed by predetermined nodes and edges, etc.) as input.

[0176] And the 2D graph encoder (f_2D) can obtain embeddings for each input node (for example, atoms, etc.) and edge (for example, union, etc.).

[0177] Additionally, in the embodiment, the two-dimensional graph encoder (f_2D) can learn representations of each node and graph level based on the obtained embeddings.

[0178] That is, the 2D graph encoder (f_2D) can learn the characteristics of each atom contained in the 2D molecular data and the overall structural characteristics of the graph (for example, molecules, etc.).

[0179] Additionally, in the embodiment, the two-dimensional graph encoder (f_2D) can learn interactions between each node of two-dimensional molecular data.

[0180] As an example, a two-dimensional graph encoder (f_2D) can learn interactions between each node using an attention mechanism.

[0181] Through this, the 2D graph encoder (f_2D) can infer complex interactions appearing in 3D molecular structures through 2D molecular graphs.

[0182] Meanwhile, from a structural perspective, the two-dimensional graph encoder (f_2D) according to the embodiment can perform data processing and learning based on an attention-based architecture (e.g., TokenGT architecture, etc.).

[0183] As an example, the two-dimensional graph encoder (f_2D) may include the TokenGT architecture, which is an attention-based architecture with maximum expressiveness for two-dimensional graphs.

[0184] Accordingly, the 2D graph encoder (f_2D) can produce consistent results regardless of the order of nodes and edges within the 2D graph structure by using all possible permutation-equivariant operators.

[0185] However, the attention-based architecture described in the embodiment is not limited to the TokenGT architecture and may include various embodiments.

[0186] Additionally, in the embodiment, the two-dimensional graph encoder (f_2D) can implement an attention mechanism according to the above attention-based architecture.

[0187] In other words, the 2D graph encoder (f_2D) can learn structural features similar to 3D even without data on 3D molecular structures by learning the interactions between each node contained in 2D molecular data through the implementation of an attention mechanism.

[0188] At this time, in the embodiment, the 2D graph encoder (f_2D) can transfer the knowledge learned from the 3D conformer encoder based on the D&D-GRAPH methodology, which is a graph-level D&D (Denoise and Distill) and the D&D-NODE methodology, which is a node-level D&D (Denoise and Distill).

[0189] Here, D&D-GRAPH according to the embodiment may be a learning method that reflects the overall structural characteristics of the graph as a method of transferring knowledge through a mean-pooled representation at the graph level.

[0190] In addition, the D&D-NODE according to the embodiment may be a learning method that more precisely reflects the characteristics of each atom, as a method for performing detailed knowledge transfer in a manner that minimizes the difference between each node representation at the node level.

[0191] As described above, the 2D graph encoder (f_2D) according to the embodiment can transfer knowledge learned from a 3D conformer encoder based on a high-performance attention-based architecture (e.g., TokenGT, etc.), and thereby perform prediction of various molecular properties using 2D molecular data with high accuracy even without 3D molecular structural information.

[0192] Depending on the embodiment, the above two-dimensional graph encoder (f_2D) may be included in the multi-task learning model described above or implemented and operated as a separate external device and / or server.

[0193] Meanwhile, according to an embodiment, the encoder module (ECM) of the multi-task learning model (MtLM) described above can be implemented based on the two-dimensional graph encoder (f_2D) as above.

[0194]

[0195] [3D Conformer Encoder (f_3D: 3D Conformer Encoder)]

[0196] A 3D conformer encoder (f_3D) according to one embodiment of the present invention may be an encoder that takes 3D molecular data (i.e., 3D structural data of a molecule) as input.

[0197] As an example, the 3D molecular data may include certain DFT data, etc.

[0198] For reference, the DFT data described above primarily refers to calculation data obtained through "density functional theory (DFT)". Density functional theory, a quantum mechanical calculation method, can be utilized to calculate the electronic structure of atoms and molecules. Such DFT data uses the electron density of a material as a key variable to calculate the total energy of a system, thereby supporting the prediction of chemical / physical properties.

[0199] Returning to the above, the 3D conformer encoder (f_3D) can perform pre-training using 3D molecular data and serve as a teacher model that transfers the knowledge learned through this to the 2D graph encoder (f_2D) described above. In other words, the 3D conformer encoder (f_3D) can be the entity that transfers knowledge to the 2D graph encoder (f_2D).

[0200] Therefore, the 3D conformer encoder (f_3D) can enable the 2D graph encoder (f_2D) to predict 3D characteristic information with high accuracy using 2D molecular data without explicit information about the 3D molecular structure.

[0201] In detail, in the embodiment, the 3D conformer encoder (f_3D) can input 3D molecular data.

[0202] And the 3D conformer encoder (f_3D) can obtain embeddings for each node (e.g., atoms, etc.) and edge (e.g., bonds, etc.) of the input 3D molecular data.

[0203] Additionally, in the embodiment, the 3D conformer encoder (f_3D) can perform pre-training using 3D molecular data based on the obtained embedding.

[0204] In more detail, the 3D conformer encoder (f_3D) can perform pre-learning to learn the spatial arrangement of each node (for example, an atom, etc.) and edge (for example, a bond, etc.) of a 3D molecular structure according to 3D molecular data.

[0205] At this time, in the embodiment, the 3D conformer encoder (f_3D) can perform denoising-based learning.

[0206] Specifically, the 3D conformer encoder (f_3D) can receive 3D molecular data (hereinafter, 3D noisy data) of a deformed structure with artificial noise (e.g., Gaussian noise, etc.) added to the 3D molecular data.

[0207] And the 3D conformer encoder (f_3D) can learn about the 3D molecular structure through the process of removing noise from the input 3D noisy data (i.e., the process of restoring it to the original data).

[0208] At this time, according to the embodiment, the 3D conformer encoder (f_3D) can perform learning that maintains SE(3) invariance.

[0209] For reference, SE(3) is an abbreviation for “Special Euclidean group in three dimensions” and can mean a special Euclidean group in three-dimensional space. This SE(3) represents a set of transformations that include rotations and translations in three-dimensional space, and can express the position and posture of an object in geometry and physics.

[0210] As an example, a 3D conformer encoder (f_3D) can perform learning based on a SE(3) transformation invariant architecture (e.g., TorchMD-NET, etc.) that learns data representations independent of rotation and translation.

[0211] That is, the 3D conformer encoder (f_3D) can extract and learn generalized knowledge that is robust to the relative positions (arrangements) of molecules in physical space by performing learning using a neural network structure with SE(3) invariance.

[0212] Additionally, in the embodiment, the 3D conformer encoder (f_3D) can transfer (distillate) the knowledge learned as above to the 2D graph encoder (f_2D).

[0213] As an example, a 3D conformer encoder (f_3D) can transfer pre-learned knowledge to a 2D graph encoder (f_2D) based on cross-modal knowledge distillation.

[0214] As described above, the 3D conformer encoder (f_3D) according to the embodiment can effectively learn the complex interactions of 3D molecular structures in a denoising manner and transfer the learned knowledge to the 2D graph encoder (f_2D).

[0215] This allows the 3D conformer encoder (f_3D) to outperform the 2D graph encoder (f_2D) in the task of predicting molecular properties without explicit information about the 3D molecular structure.

[0216] According to an embodiment, the above 3D conformer encoder (f_3D) may be included in the multi-task learning model (MtLM) described above or implemented and operated as a separate external device and / or server.

[0217]

[0218] [How to Provide a Multitasking Learning Model]

[0219] Hereinafter, a method for performing multi-tasking based on the geometric alignment of knowledge data of each task's latent space in a single integrated latent space and mutually transferring and learning the knowledge data for processing multi-tasks for outputs according to multiple domains according to an embodiment of the present invention by a computing system (1000) is described in detail.

[0220] In general, a method of performing multi-tasking based on the geometric alignment of knowledge data of each task's latent space in a single integrated latent space and transferring and learning the knowledge data of each task in order to process multi-tasks for outputs according to multiple domains, in accordance with an embodiment of the present invention, is described in detail.

[0221] In general, existing transfer learning techniques are mainly focused on classification tasks of image and / or language data sets, and have limitations in solving regression problems or problems in non-Euclidean spaces.

[0222] In particular, when the training data set is insufficient, the decline in prediction performance for the above-described problem is more inevitable, and when multitasking considering various task types is required, the performance decline is aggravated in learning and prediction for this.

[0223] Additionally, most existing methods are optimized for handling data in Euclidean space, so they do not work effectively in complex curved spaces or nonlinear spaces.

[0224] FIG. 7 illustrates an example of a conceptual diagram for explaining a multi-tasking model learning method according to one embodiment of the present invention.

[0225] Therefore, as shown in FIG. 7, a computing system (1000) according to one embodiment of the present invention aims to provide a new multi-tasking model learning method that can overcome the regression problem of a small data set and the limitations of existing transfer learning techniques, and a multi-tasking execution method using a machine learning model learned based on the method.

[0226] Hereinafter, in the description according to one embodiment of the present invention, for the sake of effective description, the material described above is limited to a ‘molecule’ and the domain thereof is described based on ‘properties’.

[0227] This is because molecular data sets typically have small amounts of data, contain diverse task types, and primarily deal with regression problems.

[0228] That is, in the case of molecular data sets, various task processing linked to numerous properties is required, but the data provided for this is very limited, and each property has the characteristic of being closely related to or influencing each other.

[0229] Considering these points, the molecular data set is advantageously applicable to multi-task processing across multiple domains, and may be a desirable example for explaining a multi-tasking model learning method according to one embodiment of the present invention and a multi-tasking performance method using a machine learning model learned based on the same.

[0230] However, it is not limited thereto, and it is obvious that any embodiment that can apply multi-tasks according to multiple domains can be included in the embodiments of the present invention.

[0231] Hereinafter, a multitasking model learning method according to one embodiment of the present invention and a multitasking performance method using a machine learning model learned based on the same will be described in more detail with reference to the attached drawings.

[0232] FIG. 8 illustrates a block flow diagram for explaining a multi-tasking model learning method according to one embodiment of the present invention.

[0233] Referring to FIG. 8, a multi-tasking model learning method according to one embodiment of the present invention and a multi-tasking performance method using a machine learning model learned based thereon may include a step (S101) of initializing a multi-tasking learning model (MtLM), a step (S103) of acquiring experimental data, a step (S105) of training the multi-tasking learning model (MtLM) based on the acquired experimental data, and a step (S107) of providing the trained multi-tasking learning model (MtLM).

[0234] In detail, a computing system (1000) according to one embodiment of the present invention can initialize a multi-tasking learning model (MtLM). (S101)

[0235] Here, in other words, the multi-task learning model (MtLM) (Geometrically Aligned Transfer Encoder Model) according to an embodiment of the present invention may be a machine learning model that mutually aligns fragmented knowledge data (for example, latent vectors, etc.) in a latent space for each task through geometric transfer in a single unified latent space (M) in order to process multiple tasks for output according to multiple domains.

[0236] In other words, the multi-tasking learning model (MtLM) according to the embodiment not only simultaneously learns knowledge data according to various domains but also efficiently learns relationships between multiple domains, thereby expanding the learning area and simultaneously performing effective multi-tasking learning that implements batch learning of local patterns according to each domain and common principles between multiple domains.

[0237] In detail, in an embodiment, the computing system (1000) can perform initialization for each component included in the multi-tasking learning model (MtLM) as described above.

[0238] As an example, a computing system (1000) includes an embedding network (MtLM) within a multi-task learning model ( ), encoder network ( ), regressor (head) network ( ), Transfer Network ( ) and / or inverse network( ) etc. random parameters ( ) can be initialized.

[0239] Additionally, as an embodiment, the computing system (1000) can set a predetermined optimization algorithm to be applied to a multi-tasking learning model (MtLM).

[0240] For example, the computing system (1000) may set the AdamW (Decoupled Weight Decay Regularization) algorithm as the optimization algorithm, and according to an embodiment, the optimization algorithm may be improved and used to independently process weight decay.

[0241] Additionally, the computing system (1000) according to one embodiment of the present invention can obtain experimental data. (S103)

[0242] Here, again, experimental data according to an embodiment of the present invention ( ) may be data including specific material-specific characteristic information and material property-specific information used for training a multi-task learning model (MtLM).

[0243] At this time, the material characteristic information according to the embodiment may be information specifying the unique characteristics possessed by a given material. That is, the material characteristic information in the embodiment may be information specifying the unique characteristics possessed by a given molecule.

[0244] For example, material characteristic information may include a given material name, molecular structure, and / or chemical formula.

[0245] In addition, the material property specific information according to the embodiment may be information that specifies the data value that a given material has for a given material property.

[0246] For example, material property specific information may include property (i.e., domain) values ​​such as boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity of a given material.

[0247] In detail, in an embodiment, the computing system (1000) can obtain experimental data as described above based on predetermined user input and / or connection with an external server.

[0248] In addition, a computing system (1000) according to one embodiment of the present invention can train a multi-task learning model (MtLM) based on acquired experimental data. (S105)

[0249] FIG. 9 illustrates a block flow diagram for explaining a multi-tasking learning model (MtLM) training method according to one embodiment of the present invention, and FIG. 10 illustrates an example of a conceptual diagram for explaining a multi-tasking learning model (MtLM) training method according to one embodiment of the present invention.

[0250] That is, referring to FIGS. 9 and 10, in the embodiment, the computing system (1000) can perform pre-learning for a multi-task learning model (MtLM) based on the experimental data acquired as described above.

[0251] In detail, in an embodiment, the computing system (1000) can set a training loop for a multi-task learning model (MtLM). (S201)

[0252] In more detail, in an embodiment, the computing system (1000) can set the number of epoch repetitions, the number of task repetitions, and / or the number of batch repetitions during training.

[0253] As an example, the computing system (1000) can set a training loop to repeatedly perform epoch 'i' from '1 to n (n>=1)' during training, repeatedly perform the same for each task 't', and repeatedly perform the same for each preset batch 'b'.

[0254] Additionally, in the embodiment, the computing system (1000) can obtain a geometric alignment vector based on the experimental data obtained as described above. (S203)

[0255] Here, the geometric alignment vector according to the embodiment of the present invention may mean various vectors obtained through a multi-task learning model (MtLM).

[0256] In the embodiment, the geometric alignment vector is an embedding vector ( ), pulturation vector ( ), encoding vectors, transition vectors, and inverse vectors.

[0257] In detail, in an embodiment, the computing system (1000) can input the acquired experimental data into a multi-task learning model (MtLM).

[0258] Additionally, in the embodiment, the computing system (1000) can obtain an embedding vector based on a multi-task learning model (MtLM) that inputs experimental data.

[0259] In more detail, the computing system (1000) can convert the input experimental data into an embedding vector through an embedding network in conjunction with the embedding module (EBM) of the multi-task learning model (MtLM).

[0260] Accordingly, the computing system (1000) can obtain an embedding vector converted into a vector format by projecting the experimental data into a predetermined embedding space.

[0261] Additionally, in the embodiment, the computing system (1000) can generate a perturbation vector based on the acquired embedding vector.

[0262] In detail, in an embodiment, the computing system (1000) can generate a plurality of perturbation vectors (i.e., perturbation points) on a predetermined periphery based on the obtained embedding vector in conjunction with the perturbation module (PBM) of the multi-task learning model (MtLM).

[0263] At this time, in the embodiment, the computing system (1000) can obtain a corresponding perturbation vector for each task by repeatedly performing the above-described functional operation for each task.

[0264] As an example, the computing system (1000) can obtain a fusion vector corresponding to task 't' and a fusion vector corresponding to task 's'.

[0265] In addition, in the embodiment, the computing system (1000) can obtain an encoding vector based on the generated perturbation vector and embedding vector.

[0266] Here, the encoding vector according to the embodiment may include a pultering latent vector, which is a latent vector generated based on a predetermined pultering vector, and an original latent vector generated based on an embedding vector, which is an original vector of the pultering vector.

[0267] In detail, in an embodiment, the computing system (1000) can convert the generated perturbation vector into a latent vector by projecting it into a latent space corresponding to the task through an encoder network in conjunction with the encoder module of a multi-task learning model (MtLM).

[0268] Additionally, in the embodiment, the computing system (1000) can convert the obtained embedding vector into a latent vector by projecting it into a latent space corresponding to the task through an encoder network in conjunction with the encoder module of the multi-task learning model (MtLM).

[0269] Thus, in the embodiment, the computing system (1000) can obtain a perturbation potential vector and an original potential vector.

[0270] At this time, in the embodiment, the computing system (1000) can repeatedly perform the above-described functional operation for each task to obtain the corresponding original potential vector and the perturbation potential vector for each task.

[0271] As an example, the computing system (1000) generates an original latent vector ( : Hereinafter, the original latent vector of the task 't' and the pulverization latent vector corresponding to the task 't' ( : Below, the jet pulverization potential vector can be obtained.

[0272] Additionally, the computing system (1000) generates an original latent vector ( : Below, the original latent vector of the task 's' and the pulverization latent vector corresponding to the task 's' ( : Below, you can obtain the potential vector of the s-pulteration.

[0273] Additionally, in the embodiment, the computing system (1000) can obtain a transition vector based on the obtained encoding vector.

[0274] Here, the transition vector according to the embodiment may include a pulverization transfer vector, which is a transition vector generated based on a predetermined pulverization potential vector, and an original transfer vector, which is a transition vector generated based on an original potential vector corresponding to the pulverization potential vector.

[0275] In detail, in an embodiment, the computing system (1000) can convert the acquired perturbation latent vector and original latent vector into a transition vector by mapping them to the latent space of another task (in an embodiment, task 's' or task 't') through a transfer network in conjunction with a transfer module (TFM) of a multi-task learning model (MtLM).

[0276] Thus, the computing system (1000) can obtain a perturbation transfer vector and an original transfer vector.

[0277] At this time, in the embodiment, the computing system (1000) can repeatedly perform the above-described functional operation for each task to obtain the corresponding original transfer vector and perturbation transfer vector for each task.

[0278] As an example, a computing system (1000) may be configured to provide an original transfer vector ( : Below, the original transfer vector t) and the perturbation transfer vector corresponding to the task 't' ( : Below, the jet pulverization transfer vector can be obtained.

[0279] Additionally, the computing system (1000) provides an original transfer vector ( : Below, the original transfer vector of s) and the transformation transfer vector corresponding to task 's' ( : Below, you can obtain the s perturbation transfer vector.

[0280] Thus, in the embodiment, the computing system (1000) can obtain geometric alignment vectors (i.e., embedding vectors, perturbation vectors, encoding vectors (including original latent vectors and perturbation latent vectors), and transfer vectors (including original transfer vectors and perturbation transfer vectors)) based on experimental data.

[0281] Additionally, in the embodiment, the computing system (1000) can obtain an inverse vector based on the acquired transition vector.

[0282] Here, the inverse vector according to the embodiment may include a pulter inverse vector, which is an inverse vector generated based on a predetermined pulter transfer vector, and an original inverse vector, which is an inverse vector generated based on an original transfer vector corresponding to the pulter transfer vector.

[0283] In detail, in an embodiment, the computing system (1000) can reconstruct the acquired perturbation transfer vector and the original transfer vector through an inverse network so that they are mapped back to the original latent space and converted into inverse vectors in conjunction with the inverse module (ITM) of the multi-task learning model (MtLM).

[0284] Thus, the computing system (1000) can obtain the perturbation inverse vector and the original inverse vector.

[0285] At this time, in the embodiment, the computing system (1000) can repeatedly perform the above-described functional operation for each task to obtain the corresponding original inverse vector and perturbation inverse vector for each task.

[0286] As an example, the computing system (1000) generates an original inverse vector ( : Below, the original inverse vector of the t and the perturbation inverse vector corresponding to the task 't' ( : Below, the jet t perturbation inverse vector can be obtained.

[0287] Additionally, the computing system (1000) generates the original inverse vector ( : Below, the original inverse vector of the task 's' and the perturbation inverse vector corresponding to the task 's' ( : Below, you can obtain the s-pulteration inverse vector.

[0288] Thus, in the embodiment, the computing system (1000) can obtain geometric alignment vectors based on experimental data (i.e., embedding vectors, perturbation vectors, encoding vectors (including original latent vectors and perturbation latent vectors), transition vectors (including original transfer vectors and perturbation transfer vectors), and inverse vectors (including original inverse vectors and perturbation inverse vectors), etc.).

[0289] Additionally, in the embodiment, the computing system (1000) can calculate a geometric alignment loss based on the obtained geometric alignment vector. (S205)

[0290] Here, the geometric alignment loss according to the embodiment of the present invention may mean various loss functions (Loss) calculated based on various vectors (i.e., geometric alignment vectors) obtained through a multi-task learning model (MtLM).

[0291] In the embodiment, the geometric alignment loss is the regression loss ( : Regression loss), autoencoder loss ( : Autoencoder loss), consistency loss( : Consistency loss), mapping loss( : Mapping loss), distance loss( : Distance loss) and / or integration loss( : Integrated loss) etc.

[0292] In the following description, for the sake of effective explanation, the geometric alignment loss is calculated based on the task 't'.

[0293] FIG. 11 and FIG. 12 illustrate examples of drawings for explaining a method for calculating regression loss according to one embodiment of the present invention.

[0294] In detail, referring to FIGS. 10 to 12, in an embodiment, a computing system (1000) can calculate a regression loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0295] In more detail, in the embodiment, the computing system (1000) predicts a predicted value () through a regression module (RGM) according to the following [Mathematical Formula 1]. ) and actual value( , i.e., the regression loss based on the label value) can be calculated. Here, the predicted value of [Mathematical Formula 1] is ' ' can also be expressed as.

[0296] [Mathematical Formula 1]

[0297]

[0298] That is, the computing system (1000) can calculate the regression loss by calculating the mean squared error (MSE) between the predicted value and the actual value.

[0299] At this time, in the embodiment, each task can prevent mutual interference by calculating an independent regression loss based on an encoder module (ECM) and a regressor module (RGM) matching each task and performing learning based on the regression loss.

[0300] In this way, the computing system (1000) can easily evaluate the regression performance of the model by calculating the regression loss.

[0301] In addition, referring further to FIG. 10, in the embodiment, the computing system (1000) can calculate the autoencoder loss2) based on a multi-task learning model (MtLM) that has obtained a geometric alignment vector.

[0302] In detail, in an embodiment, the computing system (1000) can calculate an autoencoder loss based on the original latent vector and the original inverse vector according to the following [Mathematical Formula 2].

[0303] [Mathematical Formula 2]

[0304]

[0305] That is, the computing system (1000) can calculate the autoencoder loss by calculating the mean square error (MSE) between the latent vector and the inverse vector.

[0306] In an embodiment, the computing system (1000) can improve accuracy in the data transfer process through the autoencoder loss calculated as above.

[0307] FIG. 13 illustrates an example of a drawing for explaining an integrated latent space (M) mapping method according to one embodiment of the present invention.

[0308] Meanwhile, referring to FIG. 13, in an embodiment, the computing system (1000) can learn a bidirectional transformation matrix (TM) that can be mapped to a common integrated latent space (M) for each task.

[0309] In detail, in an embodiment, the computing system (1000) can connect latent spaces between tasks by utilizing knowledge data that contain labels for both tasks.

[0310] In this process, the computing system (1000) can calculate consistency loss and mapping loss according to the embodiment.

[0311] FIG. 14 and FIG. 15 illustrate examples of drawings for explaining a consistency loss calculation method according to one embodiment of the present invention.

[0312] In more detail, referring to FIGS. 10, 14 and 15, in an embodiment, a computing system (1000) can calculate consistency loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0313] Specifically, in the embodiment, the computing system (1000) can calculate the consistency loss based on the perturbation transfer vector of task 't' and the perturbation transfer vector of task 's' according to [Mathematical Formula 3] below.

[0314] [Equation 3]

[0315]

[0316] That is, the computing system (1000) can calculate the consistency loss by calculating the mean square error (MSE) between the t-pulter transfer vector and the s-pulter transfer vector.

[0317] At this time, in the embodiment, the computing system (1000) can derive a metric for calculating a distance in space from a transformation matrix (TM), and learn to make the distance in the latent space of each task the same based on the derived metric.

[0318] Through this, the computing system (1000) can more effectively implement geometric alignment between tasks.

[0319] FIG. 16 and FIG. 17 illustrate examples of drawings for explaining a mapping loss calculation method according to one embodiment of the present invention.

[0320] In addition, referring to FIGS. 10, 16 and 17, in the embodiment, the computing system (1000) can calculate a mapping loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0321] In detail, in the embodiment, the computing system (1000) can calculate a mapping loss based on a predicted value based on an actual value according to task 't' and an original inverse vector according to task 's' according to [Mathematical Formula 4] below.

[0322] [Equation 4]

[0323]

[0324] That is, the computing system (1000) can calculate the mapping loss by calculating the mean square error (MSE) between the actual value of task 't' and the predicted value according to the original inverse vector of task 's'.

[0325] In an embodiment, the computing system (1000) can implement learning to transfer latent vectors from the latent space of one task to the latent space of the other task by calculating the mapping loss as described above, and perform the other task based on the transferred vectors, thereby inducing latent characteristics to become similar to each other.

[0326] Through this, the computing system (1000) can evaluate the prediction performance of vectors transferred to the latent space of other tasks and induce learning in a direction to improve it.

[0327] In addition, referring further to FIG. 10, in the embodiment, the computing system (1000) can calculate distance loss5) based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0328] In detail, in the embodiment, the computing system (1000) calculates the distance between the original transfer vector and the perturbation transfer vector of each task according to [Mathematical Equation 5] and [Mathematical Equation 6] below. : The distance loss between tasks can be calculated based on the transfer vector displacement.

[0329] In more detail, in the embodiment, the computing system (1000) calculates the distance between the t original transfer vector and the t perturbation transfer vector according to the task 't' according to [(a) of the mathematical expression 5] below. : Below, the t transfer vector displacement) can be calculated.

[0330] In addition, the computing system (1000) calculates the distance between the s original transfer vector and the s perturbation transfer vector according to the task 's' according to [(b) of the mathematical expression 5] below. : Below, the transfer vector displacement) can be calculated.

[0331] [Equation 5]

[0332]

[0333] Additionally, in the embodiment, the computing system (1000) can calculate the mean square error (MSE) between the t transfer vector displacement and the s transfer vector displacement according to the following [Mathematical Formula 6] to calculate the distance loss.

[0334] [Equation 6]

[0335]

[0336] Here, 'M' in [Mathematical Formula 6] means the number of pulverization points.

[0337] At this time, in the embodiment, the computing system (1000) can define the t transfer vector displacement and the s transfer vector displacement as displacements in the source task and the target task, respectively.

[0338] Thus, the computing system (1000) can more easily calculate the distance between the original transfer vector and the perturbation transfer vector by interpreting the t transfer vector displacement and the s transfer vector displacement as being in a flat Euclidean space.

[0339] Therefore, the computing system (1000) can support more complete consistency maintenance of the latent space of the model.

[0340] FIG. 18 illustrates an example of a drawing for explaining an integrated loss calculation method according to one embodiment of the present invention.

[0341] In addition, referring to FIGS. 10 and 18, in the embodiment, the computing system (1000) can calculate an integrated loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0342] In detail, in the embodiment, the computing system (1000) can calculate an integrated loss by weighting the regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss described above according to [Mathematical Formula 7] below.

[0343] [Equation 7]

[0344]

[0345] At this time, in the embodiment, the computing system (1000) can apply weights to each loss function so that each loss function can be optimized for a specific aspect of the model.

[0346] Here, ' in [Equation 7] ' is the weight of the autoencoder loss, and ' ' is the weight of consistency loss, and ' ' is the weight of the mapping loss, and ' ' is the weight of the distance loss.

[0347] In an embodiment, the computing system (1000) can update parameters in a direction to minimize the integrated loss by adjusting the importance of the loss function corresponding to each weight during the learning process of the model using the above weights.

[0348] Returning to Figure 9, the computing system (1000) in the embodiment can also perform model optimization and parameter update based on the geometric alignment loss calculated as described above. (S207)

[0349] In detail, in an embodiment, the computing system (1000) can perform optimization and parameter update for a multi-task learning model (MtLM) based on the integrated loss described above.

[0350] As an example, the computing system (1000) can calculate a gradient based on the integrated loss for each parameter of a multi-task learning model (MtLM) through backpropagation.

[0351] And the computing system (1000) can perform parameter update of a multi-task learning model (MtLM) using a calculated gradient and a preset optimization algorithm (e.g., AdamW (Decoupled Weight Decay Regularization) algorithm, etc.).

[0352] Thus, the computing system (1000) can implement a multi-task learning model (MtLM) optimization based on geometric alignment loss (particularly, integration loss).

[0353] In this way, in the embodiment, the computing system (1000) can perform multi-task learning model (MtLM) optimization and parameter update learning through a combination of multiple loss functions calculated in various ways.

[0354] At this time, each loss function can easily assist in improving the performance of the model by correcting the accuracy, consistency, and / or distance of the knowledge data mapping.

[0355] Through this, the computing system (1000) can implement a multi-tasking model that provides improved performance that overcomes the regression problem of a small data set and the limitations of existing transfer learning techniques, while operating more stably and providing improved generalization performance.

[0356] Additionally, in an embodiment, the computing system (1000) may terminate multi-task learning model (MtLM) training. (S209)

[0357] In detail, in an embodiment, the computing system (1000) can terminate the multi-task learning model (MtLM) training process described above when a preset training termination condition is met.

[0358] In an embodiment, the computing system (1000) can terminate multi-task learning model (MtLM) training upon completion of a set training loop.

[0359] Returning to Figure 8, a computing system (1000) according to an embodiment of the present invention can also provide a trained multi-task learning model (MtLM). (S107)

[0360] That is, in the embodiment, the computing system (1000) can provide a multi-task learning model (MtLM) trained as described above in a predetermined manner.

[0361] In an embodiment, the computing system (1000) may provide a multi-task learning model (MtLM) trained according to an embodiment of the present invention in conjunction with a predetermined application service (e.g., a material synthesis / evaluation service, a material property prediction service, and / or an optimal material recommendation service, etc.).

[0362] Thus, the computing system (1000) can effectively support processing of various multitasking tasks using a multitasking learning model (MtLM) with improved performance.

[0363] In this way, in the embodiment, the computing system (1000) can provide a multi-task learning model (MtLM) that provides improved performance that overcomes the regression problem of a small data set and the limitations of existing transfer learning techniques by mutually transferring and learning knowledge data of each task's latent space through geometric alignment in a single integrated latent space in order to process multiple tasks for output according to multiple domains, while operating more stably.

[0364] Through this, the computing system (1000) can provide a transfer learning-based multi-tasking model that operates stably and robustly with high generalization performance even in situations where the amount of given data is small, various task types are included, or regression problems are mainly dealt with.

[0365] In other words, the computing system (1000) can provide a multi-task learning model (MtLM) with improved prediction performance based on knowledge distilled through geometric alignment-based transfer learning performed in conjunction with other domains, even if there is a domain among multiple domains (in the embodiment, physical properties) that lacks experimental data (learning data).

[0366] For example, the computing system (1000) pre-trains a multi-task learning model (MtLM) based on the first to tenth properties for each of a plurality of molecular structural formulas, and then, when a first molecular structural formula including only data for the first to fifth properties is input, the computing system (1000) can more accurately predict data values ​​for the remaining sixth to tenth properties for the first molecular structural formula based on the knowledge data transferred and distilled through pre-training, and generate and provide output data based thereon.

[0367] In this way, the computing system (1000) according to an embodiment of the present invention can provide a multi-tasking model that implements effective transfer learning based on geometric alignment, guarantees high generalization performance, improves prediction accuracy for regression problems, supports regularization according to a combination of various loss functions, and performs a stable learning process to guarantee robust performance.

[0368] Above, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using a machine learning model learned based on the same have the effect of providing a multitasking model that maintains high performance even on a small data set by solving the problem of insufficient data by transferring knowledge learned from a source task to a target task through transfer learning.

[0369] Therefore, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using a machine learning model learned based on the same have the effect of expanding the scope of application to fields where it was difficult to apply the machine learning model due to insufficient data or domain knowledge.

[0370] In addition, the multi-tasking model learning method according to one embodiment of the present invention and the multi-tasking performance method using a machine learning model learned based thereon provide a specialized transfer learning technique that can be effectively applied to regression problems, thereby having the effect of demonstrating high prediction performance even in complex regression problems such as molecular data sets.

[0371] In addition, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using the machine learning model learned based thereon have the effect of improving the efficiency of transfer learning by maintaining geometric consistency between tasks by optimizing knowledge transfer between source tasks and target tasks through a Riemannian geometric approach.

[0372] In addition, the multi-tasking model learning method according to one embodiment of the present invention and the multi-tasking performance method using a machine learning model learned based thereon have the effect of further improving the generalization performance of the model by combining multiple loss functions to normalize various aspects of the model.

[0373] Therefore, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using the machine learning model learned based thereon provide a multitasking model that can be universally utilized for various substances (materials), thereby having the effect of improving the quality of the related industry as a whole.

[0374]

[0375] -[Method for learning the 3D geometric structure of molecules and method for predicting target properties including the same]

[0376] Hereinafter, a method for providing a model encoder training service that pre-trains a two-dimensional data-based encoder based on a D&D (Denoise and Distill) methodology that applies three-dimensional data by a computing system (1000) according to one embodiment of the present invention will be described in detail with reference to the attached drawings.

[0377] In general, existing two-dimensional molecular data-based learning techniques have limitations in achieving significant improvements in prediction performance due to molecular graph deformations that occur during the data augmentation process.

[0378] In addition, among the various properties of molecules (e.g., boiling point, melting point, surface tension and / or solubility, etc.), the three-dimensional characteristic information (e.g., molecular behavioral information, molecular physical force field information, etc.) described above can be a very important factor in processing various molecule / property prediction tasks, but it is difficult to easily estimate and predict this using existing two-dimensional molecular data-based learning techniques.

[0379] In addition, existing 3D molecular data-based learning techniques have the limitation of requiring precise coordinates of the 3D molecular structure, which results in high computational costs.

[0380] Accordingly, a computing system (1000) according to one embodiment of the present invention aims to provide a new learning method (i.e., a learning method based on the D&D framework (Denoise and Distill Framework) including a two-dimensional graph encoder (f_2D) and a three-dimensional conformer encoder (f_3D)) that can overcome performance limitations and data processing cost problems of existing learning techniques, and a task performance method including the same.

[0381] Specifically, a computing system (1000) according to an embodiment of the present invention learns generalized knowledge according to a three-dimensional molecular structure by performing denoising-based learning using three-dimensional molecular data through a three-dimensional conformer encoder (f_3D), transfers the learned knowledge to a two-dimensional graph encoder (f_2D), and optimizes the two-dimensional graph encoder (f_2D) to which the knowledge has been transferred for target properties (e.g., boiling point, melting point, and / or solubility, etc.), thereby providing a two-dimensional graph encoder (f_2D) that implements high-performance molecular property prediction without explicit three-dimensional molecular structure information.

[0382] Likewise, in the description according to one embodiment of the present invention, for the sake of effective description, the material described above is limited to a 'molecule' and the domain thereof is described based on 'properties', but is not limited thereto.

[0383] FIG. 19 illustrates a block flow diagram for explaining a method for learning a 3D geometric structure of a molecule and a method for predicting target properties including the same according to one embodiment of the present invention.

[0384] Referring to FIG. 19, a method for learning a 3D geometric structure of a molecule and a method for predicting target properties including the same according to an embodiment of the present invention may include a step (S301) of performing a first pre-training based on a 3D conformer encoder (f_3D), a step (S303) of performing a second pre-training based on a first pre-trained 3D conformer encoder (f_3D) and a 2D graph encoder (f_2D), a step (S305) of performing a third pre-training based on the second pre-trained 2D graph encoder (f_2D), and a step (S307) of providing a third pre-trained 2D graph encoder (f_2D).

[0385] In detail, a computing system (1000) according to one embodiment of the present invention can perform first pre-learning based on a three-dimensional conformer encoder (f_3D). (S301)

[0386] Figure 20 illustrates an example of a conceptual diagram for explaining the first pre-learning according to one embodiment of the present invention.

[0387] In detail, referring to FIG. 20, in an embodiment, a computing system (1000) can perform denoising-based learning (hereinafter, first pre-learning) for a 3D conformer encoder (f_3D) using a predetermined 3D molecular data set as learning data.

[0388] In more detail, in an embodiment, the computing system (1000) can insert a predetermined amount of noise into each 3D molecular data.

[0389] Specifically, in the embodiment, the computing system (1000) stores each 3D molecular data It can be expressed in the form of

[0390] Here, V represents a set of atoms, and R can represent a matrix containing the coordinates of each atom in three-dimensional space.

[0391] Additionally, in the embodiment, the computing system (1000) may generate a predetermined noise ( , for example, Gaussian noise, can be added to each 3D molecule data.

[0392] At this time, the computing system (1000) generates 3D molecular data (i.e., 3D noisy data) that is deformed as noise is added. It can be expressed in the form of

[0393] Thus, in the embodiment, the computing system (1000) can obtain a set of 3D molecular data (i.e., 3D noisy data) with a predetermined amount of noise inserted therein.

[0394] Additionally, in an embodiment, the computing system (1000) can perform denoising-based learning for a 3D conformer encoder (f_3D) using a set of 3D molecular data with inserted noise (i.e., 3D noisy data).

[0395] In detail, in an embodiment, the computing system (1000) can input each 3D noisy data as learning data into a 3D conformer encoder (f_3D).

[0396] In this way, the 3D conformer encoder (f_3D) can perform 3D molecular structure learning according to a given 3D noisy data set through a process of removing noise from the input 3D noisy data (i.e., a process of restoring it to the original data).

[0397] In more detail, in the embodiment, the 3D conformer encoder (f_3D) can perform denoising learning using 3D noisy data according to [Mathematical Formula 8] below.

[0398] [Equation 8]

[0399]

[0400] That is, in the embodiment, the 3D conformer encoder (f_3D) can perform denoising learning in a way that receives atomic coordinates with added noise as input and then predicts (restores) the original atomic coordinates.

[0401] At this time, the 3D conformer encoder (f_3D) can predict the 3D vector of each atom by adding a prediction head (h_3D) to the output end.

[0402] In addition, in the embodiment, the 3D conformer encoder (f_3D) can perform learning to minimize the difference between the noise vector predicted through each 3D noisy data and the actual noise vector according to the loss function disclosed in [Mathematical Formula 9] below.

[0403] [Equation 9]

[0404]

[0405] Here, [Equation 9] represents the distribution of a given data set and the probability distribution derived from the noise sampling procedure.

[0406] At this time, as described above, the 3D conformer encoder (f_3D) according to the embodiment has a characteristic that is invariant to rotation and translation for the SE(3) group, and through this, it can accurately learn the characteristics of the molecular structure (i.e., quantum mechanical properties) regardless of spatial transformation.

[0407] Thus, in the embodiment, the computing system (1000) can perform denoising learning based on a 3D noisy data set for a 3D conformer encoder (f_3D).

[0408] Through this, in the embodiment, the computing system (1000) can construct a noise-robust 3D conformer encoder (f_3D), and through the 3D conformer encoder (f_3D) constructed in this way, learn key information related to a 3D molecular structure more effectively and apply it to predict various molecular properties.

[0409] In addition, the computing system (1000) according to one embodiment of the present invention can perform second pre-training based on a first pre-trained 3D conformer encoder (f_3D) and a 2D graph encoder (f_2D). (S303)

[0410] Figure 21 illustrates an example of a conceptual diagram for explaining second pre-learning according to one embodiment of the present invention.

[0411] In detail, referring to FIG. 21, in an embodiment, the computing system (1000) can perform second pre-training (i.e., knowledge distillation learning) that transfers (distills) knowledge learned in a first pre-trained 3D conformer encoder (hereinafter, 3D conformer denoising encoder) to a 2D graph encoder (f_2D).

[0412] In general, pre-training using a 3D conformer encoder (f_3D) can learn generalizable 3D feature information by removing noise from the 3D molecular structure. However, in actual applications, it is very inefficient to perform expensive 3D conformer (i.e., 3D molecular data) calculations for all molecules.

[0413] Accordingly, the computing system (1000) according to the embodiment can perform cross-modal knowledge distillation learning to transfer knowledge learned in a three-dimensional conformer denoising encoder to a two-dimensional graph encoder (f_2D) in order to predict three-dimensional characteristic information with high accuracy using a two-dimensional graph encoder (f_2D).

[0414] In more detail, in an embodiment, the computing system (1000) can perform a second pre-training to train the two-dimensional graph encoder (f_2D) so that the representation output by the two-dimensional graph encoder (f_2D) follows (imitates) the representation output by the three-dimensional conformer denoising encoder.

[0415] Specifically, in the embodiment, the computing system (1000) can perform 1) graph-level knowledge transfer (D&D-GRAPH).

[0416] In detail, in an embodiment, the computing system (1000) can perform graph-level knowledge transfer in a manner that minimizes the difference between the graph representations output by a two-dimensional graph encoder (f_2D) and a three-dimensional conformer denoising encoder.

[0417] In more detail, in the embodiment, a 3D conformer denoising encoder can receive a predetermined 3D molecular data as input and learn a representation of each atom accordingly.

[0418] And the 3D conformer denoising encoder can generate a representation of the entire graph by mean pooling the learned representation.

[0419] In the same way, in the embodiment, the two-dimensional graph encoder (f_2D) can receive a given two-dimensional molecular data (G) as input, learn a representation of each atom according to the data, and average-pool the learned representations to generate a representation of the entire graph.

[0420] Thereafter, in the embodiment, the computing system (1000) can perform learning to minimize the L2 loss between the graph representation (hereinafter, the 3D average pooling graph representation) generated from the 3D conformer denoising encoder and the graph representation (hereinafter, the 2D average pooling graph representation) generated from the 2D graph encoder (f_2D) according to the loss function disclosed in [Mathematical Formula 10] below.

[0421] [Equation 10]

[0422]

[0423] Here, the pool function of [Mathematical Formula 10] refers to a function that creates a graph representation by averaging the output node representations of each encoder.

[0424] In this process, according to an embodiment, the computing system (1000) can learn and update a two-dimensional graph encoder (f_2D) while freezing a three-dimensional conformer denoising encoder. A detailed description thereof will be provided later.

[0425] Thus, in the embodiment, the computing system (1000) can perform graph-level knowledge transfer based on a 3D conformer denoising encoder and a 2D graph encoder (f_2D).

[0426] Through this, in the embodiment, the computing system (1000) can provide a framework that efficiently reduces data processing costs for predicting three-dimensional characteristic information.

[0427] Additionally, in the embodiment, the computing system (1000) can perform 2) node-level knowledge transfer (D&D-NODE).

[0428] In detail, in an embodiment, the computing system (1000) can perform node-level knowledge transfer in a manner that minimizes the difference between the node representations for each atom output by a two-dimensional graph encoder (f_2D) and a three-dimensional conformer denoising encoder.

[0429] In more detail, in the embodiment, a 3D conformer denoising encoder can receive a predetermined 3D molecular data as input and learn a representation of each atom (hereinafter, 3D atomic node representation) according to the input.

[0430] In the same way, in the embodiment, the two-dimensional graph encoder (f_2D) can receive a given two-dimensional molecular data as input and learn the representation of each atom (hereinafter, two-dimensional atom node representation) according to the data.

[0431] In the following embodiment, the computing system (1000) can perform learning to minimize the L2 loss between the three-dimensional atomic node representation and the two-dimensional atomic node representation according to the loss function disclosed in [Mathematical Formula 11] below.

[0432] [Equation 11]

[0433]

[0434] That is, in the embodiment, the 3D conformer denoising encoder and the 2D graph encoder (f_2D) each generate a unique representation for each atom, and the computing system (1000) can learn so that the outputs of each atom of the two encoders generated as described above are mutually consistent.

[0435] In this process, according to an embodiment, the computing system (1000) can learn and update a two-dimensional graph encoder (f_2D) while freezing a three-dimensional conformer denoising encoder. A detailed description thereof will be provided later.

[0436] Thus, in the embodiment, the computing system (1000) can perform node-level knowledge transfer based on a 3D conformer denoising encoder and a 2D graph encoder (f_2D).

[0437] In this way, in the embodiment, the computing system (1000) can build a two-dimensional graph encoder (f_2D) that performs predictions that more precisely reflect the knowledge transferred from the three-dimensional conformer denoising encoder by implementing knowledge transfer of physical / chemical properties to the two-dimensional graph encoder (f_2D) at a more detailed level than the graph-level knowledge transfer. Meanwhile, as described above, in the embodiment, the computing system (1000) can implement knowledge transfer that distills the knowledge learned in the three-dimensional conformer denoising encoder to the two-dimensional graph encoder (f_2D), thereby enabling the two-dimensional graph encoder (f_2D) to perform high-performance molecular property prediction that encompasses three-dimensional characteristic information using two-dimensional molecular data.

[0438] On the other hand, in the embodiment, the computing system (1000) may freeze at least some parameters of the 3D conformer denoising encoder when performing the second pre-learning (i.e., knowledge transfer learning) as described above.

[0439] The goal of the second pre-training is to transfer the knowledge learned in the 3D conformer denoising encoder to the 2D graph encoder (f_2D). To this end, the 3D conformer denoising encoder must remain frozen and fixed during the second pre-training.

[0440] This allows the 3D conformer denoising encoder to preserve previously learned information, and the 2D graph encoder (f_2D) to learn its weights based on the preserved information.

[0441] In detail, in an embodiment, the computing system (1000) can freeze at least some parameters of the 3D conformer denoising encoder in the second pre-training process to prevent updating of its weights.

[0442] FIG. 22 illustrates an example of a drawing for explaining a method for determining a model parameter freezing range during second pre-learning according to one embodiment of the present invention.

[0443] At this time, referring to FIG. 22, according to an embodiment, the computing system (1000) can determine the freezing range of the 3D conformer denoising encoder based on the amount of learning data for the second pre-learning (LDA: hereinafter, the amount of second pre-learning data).

[0444] As an example, the computing system (1000) can freeze all parameters of the 3D conformer denoising encoder if the amount of second pre-training data (LDA) is below a predetermined standard value.

[0445] On the other hand, the computing system (1000) can free all parameters of the 3D conformer denoising encoder if the amount of second pre-learning data (LDA) exceeds a predetermined standard.

[0446] In another embodiment, the computing system (1000) can determine the freezing range of the 3D conformer denoising encoder inversely proportional to the amount of second pre-training data (LDA).

[0447] That is, the computing system (1000) can narrow the freezing range of the 3D conformer denoising encoder as the amount of second pre-training data (LDA) increases, and can widen the freezing range of the 3D conformer denoising encoder as the amount of second pre-training data (LDA) decreases.

[0448] For example, the computing system (1000) can freeze a preset number of n (n>=1) parameters in the 3D conformer denoising encoder when the amount of second pre-training data (LDA) exceeds a preset first criterion and is below a preset second criterion, and can freeze a preset number of nm (m>=1) parameters in the 3D conformer denoising encoder when the amount of second pre-training data (LDA) exceeds a preset second criterion and is below a preset third criterion.

[0449] Additionally, referring further to FIG. 22, according to an embodiment, the computing system (1000) may determine the freezing range of the 3D conformer denoising encoder based on the correlation (RLV) between the 3D molecular structure and the target property according to the learning data for the second pre-learning.

[0450] In an embodiment, the computing system (1000) can determine the freezing range of a 3D conformer denoising encoder inversely proportional to the relevance (RLV) described above.

[0451] That is, the computing system (1000) can narrow the freezing range of the 3D conformer denoising encoder as the relevance (RLV) is high, and can widen the freezing range of the 3D conformer denoising encoder as the relevance (RLV) is low.

[0452] In this way, according to an embodiment, the computing system (1000) can further improve the learning performance of the second pre-learning (i.e., knowledge transfer learning) by variably determining the freezing range of the 3D conformer denoising encoder in various ways.

[0453] Returning again, the computing system (1000) can also perform the second pre-training while freezing at least a portion of the 3D conformer denoising encoder.

[0454] Thus, the computing system (1000) can implement a second pre-learning in which the parameters of the frozen 3D conformer denoising encoder are fixed while the parameters of the 2D graph encoder (f_2D) are updated.

[0455] In this way, in the embodiment, the computing system (1000) can enable the 3D conformer denoising encoder to preserve previously learned information by applying the freezing technique during the second pre-learning, and the 2D graph encoder (f_2D) can easily learn its own weights based on the preserved information.

[0456] In addition, the computing system (1000) according to one embodiment of the present invention can perform third pre-learning based on the second pre-learned two-dimensional graph encoder (f_2D). (S305)

[0457] Figure 23 illustrates an example of a conceptual diagram for explaining the third pre-learning according to one embodiment of the present invention.

[0458] In detail, referring to FIG. 23, in an embodiment, the computing system (1000) can perform third pre-training (i.e., fine-tuning learning) to optimize the second pre-trained two-dimensional graph encoder (hereinafter, two-dimensional graph transition encoder) for a given target property.

[0459] That is, in the embodiment, the computing system (1000) can fine-tune a two-dimensional graph transition encoder using a data set for each target property (e.g., boiling point, melting point, surface tension, and / or solubility, etc.) to build an encoder optimized for each target property.

[0460] In detail, in an embodiment, the computing system (1000) can collect a predetermined target property data set (e.g., an Open Graph Benchmark (OGB) data set and / or a manually constructed physical molecule property data set, etc.).

[0461] Here, the target property data set described above may include data specifying molecular structure information and / or target property values.

[0462] Additionally, in the embodiment, the computing system (1000) can perform a downstream task for a two-dimensional graph transition encoder using the collected target property data set.

[0463] For reference, downstream tasks can refer to the process of optimizing (adjusting) a pre-trained model to fit an actual application problem.

[0464] That is, in an embodiment, the computing system (1000) can perform a downstream task that optimizes a two-dimensional graph transitive encoder for a specific task, such as predicting a given target property.

[0465] As an example, the computing system (1000) can perform downstream tasks based on full model finetuning that fine-tunes all parameters of a two-dimensional graph transition encoder through a target material property data set.

[0466] In this case, the computing system (1000) can increase the possibility of deriving the best local optimum for the target property, thereby improving the prediction performance and reliability.

[0467] In another embodiment, the computing system (1000) can perform downstream tasks based on prediction head finetuning, which fine-tunes only the last prediction head of a two-dimensional graph transitive encoder through a target material property data set.

[0468] That is, the computing system (1000) can perform downstream tasks by fine-tuning only the last prediction head (i.e., the parameter that outputs the final prediction value) of the two-dimensional graph transition encoder and freezing the remaining layers.

[0469] In this case, the computing system (1000) can increase the learning speed and minimize overfitting.

[0470] At this time, in the embodiment, the computing system (1000) can perform the third pre-learning as described above using various disclosed optimization algorithms (e.g., AdamW, etc.).

[0471] In this way, in the embodiment, the computing system (1000) can build a two-dimensional graph transition encoder that internalizes knowledge optimized for a specific target property by fine-tuning the two-dimensional graph transition encoder for a given target property.

[0472] Through this, the computing system (1000) can support the two-dimensional graph transitive encoder to perform accurate and generalized predictions suitable for specific application cases.

[0473] In addition, the computing system (1000) can enable a two-dimensional graph transition encoder to output high-quality prediction values ​​for target properties even in an environment where label data is limited.

[0474] Additionally, a computing system (1000) according to one embodiment of the present invention may provide a third pre-learned two-dimensional graph encoder (f_2D). (S307)

[0475] In detail, in an embodiment, the computing system (1000) can provide a third pre-learned two-dimensional graph encoder (hereinafter, a two-dimensional graph fine-tuning encoder) according to a predetermined method.

[0476] As an example, a computing system (1000) can provide a two-dimensional graph fine-tuning encoder by applying it to a predetermined multitasking model.

[0477] For example, the computing system (1000) can provide the two-dimensional graph fine-tuning encoder described above by replacing the encoder module (ECM) of the multi-task learning model (MtLM) described above with the two-dimensional graph fine-tuning encoder.

[0478] In this way, in an embodiment, the computing system (1000) may provide a D&D (Denoise and Distill) framework that provides a two-dimensional graph encoder (f_2D) (i.e., a two-dimensional graph fine-tuning encoder) that performs high-accuracy and quality molecular property prediction even in the absence of explicit three-dimensional molecular structure information.

[0479] Thus, in the embodiment, the computing system (1000) can directly improve the overall quality and performance of related industries and services.

[0480] A method for learning a 3D geometric structure of a molecule and a method for predicting target properties including the same according to one embodiment of the present invention have the effect of enabling a 2D level encoder to efficiently include 3D level information by pre-training a 2D data-based encoder based on a D&D (Denoise and Distill) framework that applies 3D data.

[0481] Accordingly, the method for learning the 3D geometric structure of a molecule and the method for predicting target properties including the same according to one embodiment of the present invention have the effect of providing a high-performance encoder that performs molecular property prediction based on 3D level information through 2D data (e.g., 2D molecular graph, etc.).

[0482] Thus, the method for learning the 3D geometric structure of a molecule according to one embodiment of the present invention and the method for predicting target properties including the same have the effect of significantly reducing the computational cost while achieving high prediction accuracy by utilizing 3D level information.

[0483] In addition, a method for learning a 3D geometric structure of a molecule according to an embodiment of the present invention and a method for predicting target properties including the same perform denoising-based learning through a 3D conformer encoder (f_3D) and then perform learning to transfer (distillate) the knowledge learned in the 3D conformer encoder (f_3D) to a 2D graph encoder (f_2D), thereby providing a high-performance encoder that combines the strengths of 3D-based molecular representation learning and the efficiency of 2D-based molecular representation learning.

[0484] In addition, according to one embodiment of the present invention, a method for learning a 3D geometric structure of a molecule and a method for predicting target properties including the same implement a two-dimensional level encoder that effectively learns generalized knowledge from a given data set, thereby having the effect of maintaining high prediction performance even in an environment where label data is limited.

[0485] In addition, the method for learning the 3D geometric structure of a molecule according to one embodiment of the present invention and the method for predicting target properties including the same have the effect of performing learning to optimize (fine-tune) a 2D graph encoder (f_2D) that has received knowledge for a specific property, thereby performing accurate and generalized prediction suitable for a specific application case.

[0486] In addition, the method for learning the 3D geometric structure of a molecule according to one embodiment of the present invention and the method for predicting target properties including the same have the effect of improving the efficiency of learning and prediction for large-scale data while simultaneously increasing its versatility and expandability by providing the D&D (Denoise and Distill) framework as described above.

[0487] That is, the method for learning the 3D geometric structure of a molecule according to one embodiment of the present invention and the method for predicting target properties including the same provide a generalized learning framework that can be applied to various molecular science problems, not limited to specific molecular property prediction problems, thereby providing an improved encoder that can be widely utilized in various research and industrial applications.

[0488]

[0489] Meanwhile, the embodiments of the present invention described above may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. Hardware devices may be changed into one or more software modules to perform processing according to the present invention, and vice versa.

[0490] The specific implementations described in the present invention are exemplary embodiments and do not limit the scope of the present invention in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the lines connecting or connecting members between components illustrated in the drawings are merely representative of functional connections and / or physical or circuit connections, and may be replaced or represented as various additional functional connections, physical connections, or circuit connections in an actual device. In addition, unless specifically mentioned as “essential,” “important,” etc., a component may not be absolutely necessary for the application of the present invention.

[0491] Although the detailed description of the present invention has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art or having ordinary knowledge in the art that various modifications and changes can be made to the present invention without departing from the spirit and technical scope of the present invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

[0492]

[0493] The present invention relates to a multitasking model learning method and a multitasking performance method using a machine learning model learned based on the same, and is applicable to the artificial intelligence industry, and thus has industrial applicability.

Claims

1. A method for a computing system including a memory and a processor to learn the 3D geometric structure of a molecule and predict target properties, A step of performing a first pre-learning based on denoising based on a 3D Conformer Encoder that inputs 3D molecular data, which is data that specifies a 3D molecular structure; A step of performing a second pre-learning based on knowledge distillation based on the first pre-learned 3D conformer encoder and a 2D graph encoder that inputs 2D molecular data, which is data specifying a 2D level molecular structure; A step of performing a third pre-training based on fine-tuning based on the second pre-trained two-dimensional graph encoder; and comprising a step of providing the third pre-learned two-dimensional graph encoder; A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

2. In paragraph 1, The step of performing the above first pre-learning is as follows: A step of inserting a predetermined amount of noise into the above 3D molecular data, A step of restoring the 3D molecular data with the above noise inserted into it to the original 3D molecular data and training the 3D conformer encoder to learn the 3D level molecular structure. A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

3. In paragraph 2, The step of performing the above first pre-learning is as follows: Further comprising a step of learning data representation independent of rotation and translation in three-dimensional space based on a predetermined SE(3) transformation invariant architecture. A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

4. In paragraph 1, The step of performing the above second pre-learning is: A step of performing distillation learning using the 3D Conformer Denoising Encoder, which is the first pre-learned 3D conformer encoder, as a teacher model and the 2D graph encoder as a student model. A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

5. In paragraph 4, The steps for performing the above transfer learning are: A step of training the two-dimensional graph encoder so that the output representation of the two-dimensional graph encoder follows the output representation of the three-dimensional conformer denoising encoder. A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

6. In paragraph 5, The steps for performing the above transfer learning are: Further comprising a step of performing graph-level knowledge transfer (D&D-GRAPH) to minimize the difference between the graph-level representation output by the two-dimensional graph encoder and the graph-level representation output by the three-dimensional conformer denoising encoder. A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

7. In paragraph 5, The steps for performing the above transfer learning are: Further comprising a step of performing node-level knowledge transfer (D&D-NODE) ​​to minimize the difference between the node-level representation output by the two-dimensional graph encoder and the node-level representation output by the three-dimensional conformer denoising encoder. A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

8. In paragraph 5, The steps for performing the above transfer learning are: Further comprising a step of freezing at least some parameters of the above 3D conformer denoising encoder. A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

9. In paragraph 1, The step of performing the above third pre-learning is: A step of performing a downstream task of optimizing the 2D graph transition encoder, which is the second pre-learned 2D graph encoder, to a predetermined property. A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

10. In paragraph 1, The step of providing the third pre-learned two-dimensional graph encoder is: A step of applying the 2D graph fine-tuning encoder, which is the third pre-learned 2D graph encoder, to a predetermined multitasking model. A method for learning the 3D geometric structure of a molecule and a method for predicting target properties including the same.

11. At least one memory; and At least one processor for learning the 3D geometric structure of a molecule and predicting target properties by reading out at least one application stored in the memory; The instructions of the above processor are: A step of performing a first pre-learning based on denoising based on a 3D Conformer Encoder that inputs 3D molecular data, which is data that specifies a 3D molecular structure, A step of performing a second pre-learning based on knowledge distillation based on the first pre-learned 3D conformer encoder and a 2D graph encoder that inputs 2D molecular data, which is data specifying a 2D level molecular structure, A step of performing a third pre-learning based on fine-tuning based on the above second pre-learned two-dimensional graph encoder, A command comprising a step of providing the third pre-learned two-dimensional graph encoder. A system for learning 3D geometric structures of molecules and predicting target properties.

Images