Multi-tasking model training method and multi-tasking performing method using machine learning model trained on basis thereof

Abstract

A multi-tasking model training method and a multi-tasking performing method using a machine learning model trained on the basis thereof, according to an embodiment of the present invention, may mutually transfer and learn knowledge data of a latent space for each task through geometric alignment in one integrated latent space in order to process a multi-task for output according to a plurality of domains.

Description

A method for learning a multitasking model and a method for performing multitasking using a machine learning model learned based on the method.

[0001] The present invention relates to a multi-tasking model learning method and a multi-tasking execution method using a machine learning model learned based on the same. More specifically, the present invention relates to a multi-tasking model learning method that mutually transfers and learns knowledge data in each task's latent space through geometric alignment in a single, integrated latent space, in order to process multiple tasks for output according to multiple domains, and to a multi-tasking execution method using a machine learning model learned based on the same.

[0002] Machine learning and artificial intelligence models require massive amounts of data. However, realistically, there are always limitations to acquiring sufficient data. This becomes even more acute when applying models to new domains or tasks. Molecular structure datasets are a prime example of this situation. In the fields of chemistry and pharmacology, data is needed to predict the properties of new molecules, but experimental data for each molecule is difficult and expensive to obtain. Therefore, the need for transfer learning techniques, which apply knowledge from previously trained models to new tasks, has increased.

[0003] However, existing transfer learning has primarily focused on classification problems in large-scale datasets, such as images or text data. Therefore, existing transfer learning techniques have limitations when applied to small, complex datasets, such as regression or molecular datasets. In particular, for high-dimensional problems like molecular structure data, where the relationships between individual components and combinations are crucial, existing Euclidean-based transfer learning techniques cannot effectively handle the complex structures in non-Euclidean spaces.

[0004] Meanwhile, Riemannian geometry enables calculus in curved spaces, allowing for better representation and analysis of the complex structure of data. This Riemannian geometry approach assumes that latent vectors exist on a curved manifold, which is advantageous for aligning the geometries between source and target tasks.

[0005] Therefore, based on the above background, it is necessary to introduce a new technology that can demonstrate high prediction performance and stability even on small data sets, implement more effective transfer learning, and improve model normalization performance to enhance generalization performance.

[0006] One embodiment of the present invention provides a multi-task model learning method that mutually transfers and learns knowledge data of each task's latent space through geometric alignment in a single integrated latent space in order to process multi-tasks for output according to multiple domains, and a multi-task performance method using a machine learning model learned based on the method.

[0007] However, the technical problems to be solved by the present invention and embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

[0008] A method for learning a multi-tasking model according to an embodiment of the present invention is a method for a computing system including a memory and a processor to learn a multi-tasking model, the method comprising: initializing a multi-tasking model that processes multiple tasks based on a plurality of domains; acquiring predetermined experimental data; and training the multi-tasking model based on the acquired experimental data, wherein the training of the multi-tasking model includes: acquiring a geometric alignment vector, which is a vector that supports geometric alignment between data in a single integrated latent space, based on the experimental data; calculating a geometric alignment loss based on the acquired geometric alignment vector; and updating parameters of the multi-tasking model based on the calculated geometric alignment loss.

[0009] In another aspect, the step of obtaining the geometric alignment vector includes the step of obtaining an embedding vector converted into a vector format by projecting the experimental data into a predetermined embedding space based on the embedding module included in the multitasking model.

[0010] In another aspect, the step of obtaining the geometric alignment vector further includes the step of obtaining a plurality of perturbation vectors that move the embedding vector in a predetermined direction based on a perturbation module included in the multitasking model.

[0011] In another aspect, the step of obtaining the geometric alignment vector further includes the step of obtaining an original latent vector by projecting the embedding vector onto the latent space of the first task (Task 1) based on the encoder module included in the multi-tasking model, and the step of obtaining a perturbation latent vector by projecting the perturbation vector onto the latent space of the first task based on the encoder module included in the multi-tasking model.

[0012] In another aspect, the step of obtaining the geometric alignment vector further includes the step of obtaining an original transfer vector that maps the original latent vector to the latent space of the second task (Task 2) based on a transfer module included in the multitasking model, and the step of obtaining a perturbation transfer vector that maps the perturbation latent vector to the latent space of the second task based on the transfer module included in the multitasking model.

[0013] In another aspect, the step of obtaining the geometric alignment vector further includes the step of obtaining an original inverse vector obtained by remapping the original transfer vector to the latent space of the first task based on an inverse transfer module included in the multitasking model, and the step of obtaining a perturbation inverse vector obtained by remapping the perturbation transfer vector to the latent space of the first task based on the inverse module included in the multitasking model.

[0014] In another aspect, the step of calculating the geometric alignment loss includes the step of calculating a regression loss, an autoencoder loss, a consistency loss, a mapping loss, and a distance loss based on the geometric alignment vector, and the step of calculating an integrated loss by weighting the calculated regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss.

[0015] In another aspect, the step of updating the parameters of the multitasking model includes the step of updating the parameters in a direction that minimizes the integration loss.

[0016] In another aspect, the step of calculating the geometric alignment loss further includes the step of obtaining a prediction value according to the latent vector based on a regressor module included in the multi-task model, the step of calculating a mean squared error based on the obtained prediction value and a label value corresponding to the latent vector, and the step of calculating the regression loss based on the calculated mean squared error.

[0017] In another aspect, the step of calculating the geometric alignment loss further includes the step of calculating a mean square error based on the original latent vector and the original inverse vector, and the step of calculating the autoencoder loss based on the calculated mean square error.

[0018] In another aspect, the step of calculating the geometric alignment loss further includes the step of calculating a mean square error based on a perturbation transfer vector mapped from the latent space of the first task to the latent space of the second task and a perturbation transfer vector mapped from the latent space of the second task to the latent space of the first task, and the step of calculating the consistency loss based on the calculated mean square error.

[0019] In another aspect, the step of calculating the geometric alignment loss further includes the step of calculating a mean square error based on a label value based on the first task and a predicted value according to an original inverse vector based on the second task, and the step of calculating the mapping loss based on the calculated mean square error.

[0020] In another aspect, the step of calculating the geometric alignment loss further includes the steps of calculating a first transfer vector displacement, which is a distance between an original transfer vector and a pultered transfer vector based on the first task, the step of calculating a second transfer vector displacement, which is a distance between the original transfer vector and the pultered transfer vector based on the second task, the step of calculating a mean square error based on the calculated first transfer vector displacement and the second transfer vector displacement, and the step of calculating the distance loss based on the calculated mean square error.

[0021] In another aspect, the above experimental data includes material-specific characteristic information, which is information specifying the unique characteristics possessed by a given material, and material property-specific information, which is information specifying the data value possessed by a given material for a given physical property.

[0022] Meanwhile, a multi-tasking model learning server according to an embodiment of the present invention includes at least one memory; and at least one processor for learning a multi-tasking model by reading at least one application stored in the memory, wherein instructions of the processor include instructions for performing a step of initializing a multi-tasking model that processes multiple tasks based on a plurality of domains, a step of acquiring predetermined experimental data, a step of acquiring a geometric alignment vector, which is a vector that supports geometric alignment between data in a single integrated latent space, based on the acquired experimental data, a step of calculating a geometric alignment loss based on the acquired geometric alignment vector, and a step of updating parameters of the multi-tasking model based on the calculated geometric alignment loss.

[0023] A multitasking model learning method according to one embodiment of the present invention and a multitasking performance method using a machine learning model learned based thereon have the effect of providing a multitasking model that maintains high performance even in a small data set by solving the problem of insufficient data by transferring knowledge learned in a source task to a target task through transfer learning.

[0024] Therefore, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using a machine learning model learned based on the same have the effect of expanding the scope of application to fields where it was difficult to apply the machine learning model due to insufficient data or domain knowledge.

[0025] In addition, the multi-tasking model learning method according to one embodiment of the present invention and the multi-tasking performance method using a machine learning model learned based thereon provide a specialized transfer learning technique that can be effectively applied to regression problems, thereby having the effect of demonstrating high prediction performance even in complex regression problems such as molecular data sets.

[0026] In addition, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using the machine learning model learned based thereon have the effect of improving the efficiency of transfer learning by maintaining geometric consistency between tasks by optimizing knowledge transfer between source tasks and target tasks through a Riemannian geometric approach.

[0027] In addition, the multi-tasking model learning method according to one embodiment of the present invention and the multi-tasking performance method using a machine learning model learned based thereon have the effect of further improving the generalization performance of the model by combining multiple loss functions to normalize various aspects of the model.

[0028] Therefore, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using the machine learning model learned based thereon provide a multitasking model that can be universally utilized for various substances (materials), thereby having the effect of improving the quality of the related industry as a whole.

[0029] However, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood from the description below.

[0030] FIG. 1 illustrates an example of a block diagram of a computing system that implements a multi-tasking learning model provision service according to one embodiment of the present invention.

[0031] FIG. 2 illustrates an example block diagram of a computing device that implements a multi-tasking learning model provision service according to one embodiment of the present invention.

[0032] FIG. 3 illustrates an example block diagram from another aspect of a computing device implementing a multi-tasking learning model providing service according to one embodiment of the present invention.

[0033] Figures 4 and 5 illustrate examples of conceptual diagrams for explaining a multitasking learning model according to one embodiment of the present invention.

[0034] Figure 6 illustrates an internal block diagram of a multitasking learning model according to one embodiment of the present invention.

[0035] FIG. 7 illustrates an example of a conceptual diagram for explaining a multi-tasking model learning method according to one embodiment of the present invention.

[0036] FIG. 8 illustrates a block flow diagram for explaining a multi-tasking model learning method according to one embodiment of the present invention.

[0037] FIG. 9 illustrates a block flow diagram for explaining a multi-tasking learning model training method according to one embodiment of the present invention.

[0038] FIG. 10 illustrates an example of a conceptual diagram for explaining a multi-tasking learning model training method according to one embodiment of the present invention.

[0039] FIG. 11 and FIG. 12 illustrate examples of drawings for explaining a method for calculating regression loss according to one embodiment of the present invention.

[0040] FIG. 13 illustrates an example of a drawing for explaining an integrated latent space mapping method according to one embodiment of the present invention.

[0041] FIG. 14 and FIG. 15 illustrate examples of drawings for explaining a consistency loss calculation method according to one embodiment of the present invention.

[0042] FIG. 16 and FIG. 17 illustrate examples of drawings for explaining a mapping loss calculation method according to one embodiment of the present invention.

[0043] FIG. 18 illustrates an example of a drawing for explaining an integrated loss calculation method according to one embodiment of the present invention.

[0044] The present invention is capable of various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, as well as the methods for achieving them, will become clear with reference to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms. In the following embodiments, the terms "first," "second," etc. are not used in a limiting sense but are used for the purpose of distinguishing one component from another. Furthermore, the singular expression includes the plural expression unless the context clearly indicates otherwise. Furthermore, terms such as "include" or "have" indicate the presence of a feature or component described in the specification, and do not preemptively exclude the possibility that one or more other features or components may be added. Furthermore, in the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to what is shown.

[0045] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals and redundant descriptions thereof will be omitted.

[0046]

[0047] [Example system implementing a multitasking learning model provisioning service]

[0048] Hereinafter, an exemplary system for implementing a multi-task learning model service that performs multi-tasking based on geometric alignment of knowledge data of each task's latent space in a single integrated latent space and mutually transfers and learns the knowledge data for multi-tasks for output according to multiple domains is described in detail with reference to the attached drawings.

[0049] FIG. 1 illustrates an example of a block diagram of a computing system that implements a multi-tasking learning model provision service according to one embodiment of the present invention.

[0050] Referring to FIG. 1, a computing system (1000) implementing a multi-tasking learning model provision service of the present invention includes a user computing device (110), a server computing system (130), and a training computing system (150), and the devices are capable of communicating via a network (170).

[0051] A multitasking model learning method according to one embodiment of the present invention and a multitasking performance method using a machine learning model learned based thereon may be implemented and provided locally by a user computing device (110), implemented and provided locally by a server computing system (130) communicating with the user computing device (110) in the form of a web service, or implemented and provided by a user computing device (110) and a server computing system (130) in connection with each other.

[0052] At this time, in the embodiment, the user computing device (110) and / or the server computing system (130) can train the machine learning model (120 and / or 140) through interaction with a training computing system (150) communicatively connected via a network (170). The training computing system (150) may be separate from the server computing system (130) or may be part of the server computing system (130).

[0053] And at this time, the artificial intelligence model can be 1) trained directly locally by the user computing device (110), 2) trained by the server computing system (130) and the user computing device (110) interacting with each other through a network (170), and 3) trained by a separate training computing system (150) using various training techniques and learning techniques. And the artificial intelligence model trained by the training computing system (150) can be implemented in a manner of being provided / updated by transmitting the model to the user computing device (110) and / or the server computing system (130) through a network (170).

[0054] In some embodiments, the training computing system (150) may be part of a server computing system (130) or part of a user computing device (110).

[0055] The user computing device (110) may include any type of computing device, such as a smart phone, a mobile phone, a digital broadcasting device, a personal digital assistant (PDA), a portable multimedia player (PMP), a desktop, a wearable device, an embedded computing device, and / or a tablet PC.

[0056] Such a user computing device (110) includes at least one processor (111) and memory (112). Here, the processor (111) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0057] The memory (112) may include one or more non-transitory / transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof, and may include web storage of a server that performs a memory storage function on the Internet. The memory (112) may store data (113) and commands (114) necessary for the at least one processor (111) to perform functional operations, such as training an artificial intelligence model or executing multi-tasking learning through an artificial intelligence model.

[0058] In one embodiment, the user computing device (110) may store at least one machine learning model (120).

[0059] In detail, the machine learning model (120) may be various machine learning models such as multiple neural networks (e.g., deep neural networks) or other types of machine learning models including nonlinear models and / or linear models, and may be composed of a combination thereof.

[0060] At this time, the neural network may include at least one of feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, and / or other types of neural networks.

[0061] In one embodiment, the user computing device (110) may receive at least one machine learning model (120) from a server computing system (130) via a network (170), store the model in a memory (112), and then execute the stored machine learning model (120) by a processor (111) to perform multi-tasking learning, etc.

[0062] In another embodiment, the server computing system (130) may include at least one machine learning model (140), perform operations through the machine learning model (140), and communicate data related thereto with the user computing device (110) in conjunction with the user computing device (110) to provide a multi-tasking learning model provision service to the user.

[0063] For example, a user computing device (110) can perform a multi-tasking learning model provision service in a manner that a server computing system (130) provides output for a user's input using a machine learning model (140) via the web.

[0064] Additionally, the artificial intelligence model may be implemented in such a way that at least some of the machine learning models (120 and / or 140) are executed on a user computing device (110) and the rest are executed on a server computing system (130).

[0065] In addition, the user computing device (110) may include at least one input component (121) that detects a user's input. For example, the user input component (121) may include a touch sensor (e.g., a touch screen and / or a touch pad, etc.) that detects a touch of a user's input medium (e.g., a finger or a stylus), an image sensor that detects a user's motion input, a microphone, a button, a mouse, and / or a keyboard that detects a user's voice input, etc. In addition, the user input component (121) may include an interface and an external controller when receiving an input to an external controller (e.g., a mouse and / or a keyboard, etc.) through an interface.

[0066] The server computing system (130) includes at least one processor (131) and memory (132). Here, the processor (131) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0067] And the memory (132) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. This memory (132) may store data (133) and instructions (134) necessary for the processor (131) to perform functional operations such as training an artificial intelligence model or executing multi-tasking learning through an artificial intelligence model.

[0068] In one embodiment, the server computing system (130) may be implemented to include at least one computing device. For example, the server computing system (130) may be implemented to operate multiple computing devices according to a sequential computing architecture, a parallel computing architecture, or a combination thereof. Additionally, the server computing system (130) may include multiple computing devices connected via a network (170).

[0069] Additionally, the server computing system (130) may store at least one machine learning model (140). For example, the server computing system (130) may include a neural network and / or other multi-layer nonlinear model as the machine learning model (140). Exemplary neural networks may include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0070] The training computing system (150) includes at least one processor (151) and memory (152). Here, the processor (151) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0071] And the memory (152) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. This memory (152) may store data (153) and instructions (154) necessary for the processor (151) to perform learning of an artificial intelligence model, etc.

[0072] For example, the training computing system (150) may include a model trainer (160) that trains a machine learning model (120 and / or 140) stored in the user computing device (110) and / or server computing system (130) using various training or learning techniques, such as backpropagation of errors (according to the framework illustrated in FIG. 3).

[0073] For example, such a model trainer (160) may perform updates to one or more parameters of a machine learning model (120 and / or 140) in a backpropagation manner based on a defined loss function.

[0074] In some implementations, performing backward propagation of errors may include performing truncated backpropagation through time. The model trainer (160) may perform a number of generalization techniques (e.g., weight reduction, dropout, and / or knowledge distillation) to improve the generalization ability of the trained machine learning model (120 and / or 140).

[0075] In particular, the model trainer (160) can train the machine learning model (120 and / or 140) based on a series of training data (161). Here, the training data (161) can include data in different formats, such as, for example, images, audio samples, and / or text. Examples of image types that can be used can include video frames, LiDAR point clouds, X-ray images, computed tomography scans, hyperspectral images, and / or various other forms of images.

[0076] Such training data (161) may be provided by a user computing device (110) and / or a server computing system (130). When the training computing device trains a machine learning model (120 and / or 140) on specific data of the user computing device (110), the machine learning model (120 and / or 140) may be characterized as a personalized model.

[0077] And the model trainer (160) includes computer logic utilized to provide the desired function.

[0078] Additionally, the model trainer (160) may be implemented as hardware, firmware, and / or software that controls a general-purpose processor. In one implementation, the model trainer (160) includes a program file stored in a storage device, which may be loaded into a memory (152) and executed by one or more processors (151). In another implementation, the model trainer (160) includes one or more sets of computer-executable data (153) and instructions (154) stored in a tangible computer-readable storage medium, such as a RAM hard disk or an optical or magnetic medium.

[0079] The network (170) includes, but is not limited to, a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), a Wireless Local Area Network (Wireless LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, and / or a Digital Multimedia Broadcasting (DMB) network.

[0080] In general, communication over a network (170) may be performed using any type of wired and / or wireless connection, using various communication protocols (e.g., TCP / IP, HTTP, SMTP, and / or FTP), encodings or formats (e.g., HTML and / or XML), and / or protection schemes (e.g., VPN, Secure HTTP, and / or SSL).

[0081] FIG. 2 illustrates an example block diagram of a computing device that implements a multi-tasking learning model provision service according to one embodiment of the present invention.

[0082] As shown in FIG. 2, the computing device (100) included in the user computing device (110), the server computing system (130), and the training computing system (150) includes a plurality of applications (e.g., Application 1 to Application N). Each application may include a machine learning library and one or more machine learning models. For example, the applications may include an image processing (e.g., Detection, Classification, and / or Segmentation, etc.) application, a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and / or a chat-bot application.

[0083] In an embodiment, the computing device (100) may include a model trainer (160) for training an artificial intelligence model, and may store and operate the trained artificial intelligence model to provide output data according to predetermined input data (for example, material characteristic information and / or material property information, etc.).

[0084] Each application of the computing device (100) may communicate with a number of other components of the computing device (100), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In one embodiment, each application may communicate with each device component using an API (e.g., a public API). In one embodiment, the API used by each application may be specific to that application.

[0085] FIG. 3 illustrates an example block diagram of another aspect of a computing device implementing a multi-tasking learning model providing service according to one embodiment of the present invention.

[0086] Referring to FIG. 3, a computing device (200) includes a plurality of applications (e.g., Application 1 to Application N). Each application may communicate with a central intelligence layer. For example, the applications may include an image processing application, a text messaging application, an email application, a dictation application, a virtual keyboard application, and / or a browser application. In one embodiment, each application may communicate with the central intelligence layer (and the models stored therein) using an API (e.g., a common API across all applications).

[0087] The central intelligence layer may include multiple machine learning models. For example, as illustrated in FIG. 3, at least some of the machine learning models may be provided to each application and managed by the central intelligence layer. In other implementations, two or more applications may share a single machine learning model. For example, in some implementations, the central intelligence layer may provide a single model to all applications. In some implementations, the central intelligence layer may be included within the operating system of the computing device (200) or implemented differently.

[0088] The central intelligence layer may communicate with a central device data layer. The central device data layer may be a centralized data repository for the computing device (200). As illustrated in FIG. 3, the central device data layer may communicate with a number of other components of the computing device (200), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer may communicate with each device component using an API (e.g., a private API).

[0089] The techniques described herein may refer to servers, databases, software applications, and other computer-based systems, as well as actions taken and information transmitted to or from such systems. It will be appreciated that the inherent flexibility of computer-based systems allows for a wide range of possible configurations, combinations, and division of labor and functionality between and among components. For example, the processes described herein may be implemented using a single device or component, or multiple devices or components operating in combination. Databases and applications may be implemented on a single system or in a distributed system across multiple systems. Distributed components may operate sequentially or in parallel.

[0090]

[0091] Multitasking Learning Model (MtLM)

[0092] FIGS. 4 and 5 illustrate examples of conceptual diagrams for explaining a multi-task learning model (MtLM) according to one embodiment of the present invention.

[0093] Referring to FIGS. 4 and 5, a multi-task learning model (MtLM) (Geometrically Aligned Transfer Encoder Model) according to an embodiment of the present invention may be a machine learning model that aligns fragmented knowledge data (for example, latent vectors, etc.) in each task's latent space through geometric transfer in a single integrated latent space (M: Manifold) in order to process multi-tasks for output according to multiple domains.

[0094] In other words, the multi-tasking learning model (MtLM) according to the embodiment not only simultaneously learns knowledge data according to various domains but also efficiently learns relationships between multiple domains, thereby expanding the learning area and simultaneously performing effective multi-tasking learning that implements batch learning of local patterns according to each domain and common principles between multiple domains.

[0095] Accordingly, the multi-tasking learning model (MtLM) can directly improve the processing performance and accuracy of various multi-tasking tasks based on the model learned as above.

[0096] In an embodiment, such a multi-task learning model (MtLM) can perform pre-training based on certain experimental data.

[0097] Here, the experimental data according to the embodiment may be data including predetermined material-specific characteristic information and material property-specific information as learning data used for training a multi-task learning model (MtLM).

[0098] At this time, the material characteristic information according to the embodiment may be information that specifies the unique characteristics possessed by a given material.

[0099] For example, material characteristic information may include a given material name, molecular structure, and / or chemical formula.

[0100] In addition, the material property specific information according to the embodiment may be information that specifies the data value that a given material has for a given material property.

[0101] For example, material property specific information may include property (i.e., domain) values ​​such as boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity of a given material.

[0102] Meanwhile, the multi-task learning model (MtLM) that performed pre-learning as described above in the embodiment can input predetermined material-specific characteristic information and / or material property-specific information, and output predicted data based on the input information and learned knowledge.

[0103] As an example, a multi-task learning model (MtLM) can input certain material-specific characteristic information and output predicted material property-specific information based on the input information and learned knowledge.

[0104] In another embodiment, a multi-task learning model (MtLM) can input certain material property information and output predicted material property information based on the input information and learned knowledge.

[0105] In another embodiment, a multi-task learning model (MtLM) can input predetermined material-specific characteristic information and material-specific property information, and output optimal material-specific characteristic information and material-specific property information predicted based on the input information and learned knowledge.

[0106] FIG. 6 illustrates an internal block diagram of a multi-task learning model (MtLM) according to one embodiment of the present invention.

[0107] Referring to FIG. 6, in another aspect, a multi-task learning model (MtLM) according to an embodiment may include at least one embedding module (EBM: Embedding Module), an encoder module (ECM: Encoder Module), a regressor module (RGM: Regressor Module), a transfer module (TFM: Transfer Module), an inverse transfer module (ITM: Inverse Transfer Module), a perturbation module (PBM: Perturbation Module), and a loss calculation module (LCM: Loss Calculation Module).

[0108] In detail, the embedding module (EBM) according to an embodiment of the present invention may be a pre-encoder module that converts a predetermined input data into an embedding vector.

[0109] In other words, the embedding module (EBM) may be a module that converts specific input data into a vector format by projecting it into a predetermined embedding space.

[0110] As an example, the embedding module (EBM) can provide an embedding vector for input data based on a Directed Message Passing Neural Network (DMPNN) structure.

[0111] In addition, the encoder module (ECM) according to an embodiment of the present invention may be a module that takes a predetermined embedding vector as input and converts the input embedding vector into a latent vector by projecting it into a latent space corresponding to the task.

[0112] In other words, the encoder module (ECM) can be a module that extracts the main features of the input embedding vector and expresses them on the corresponding latent space.

[0113] In an embodiment, such an encoder module (ECM) may include a plurality of encoder modules (ECMs) corresponding to each of a plurality of domains.

[0114] In an embodiment, the encoder module (ECM) may include a first encoder module (ECM) corresponding to a first domain (e.g., boiling point) and a second encoder module (ECM) corresponding to a second domain (e.g., melting point).

[0115] At this time, in the embodiment, one of the plurality of encoder modules (ECMs) may be a source encoder module (ECM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0116] In addition, any one of the remaining encoder modules (ECMs) excluding the source encoder module (ECM) may be a target encoder module (ECM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0117] In addition, the regressor module (RGM) according to an embodiment of the present invention may be a head module that takes a predetermined latent vector as input and generates a final prediction value according to the input latent vector.

[0118] These regression modules (RGMs) can directly participate in generating the final output and thus determine the model's predictive performance.

[0119] Additionally, in the embodiment, the regressor module (RGM) may include a plurality of regressor modules (RGMs) corresponding to each of a plurality of domains.

[0120] In an embodiment, the regressor module (RGM) may include a first regressor module (RGM) corresponding to a first domain (e.g., boiling point) and a second regressor module (RGM) corresponding to a second domain (e.g., melting point).

[0121] At this time, in the embodiment, one of the plurality of regressor modules (RGMs) may be a source regressor module (RGM), which is a regressor module (RGM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0122] In addition, any one of the remaining regressor modules (RGMs) excluding the source regressor module (RGM) may be a target regressor module (RGM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0123] In addition, a transfer module (TFM) according to an embodiment of the present invention may be a module that converts a predetermined latent vector into a transfer vector by mapping it to a latent space of another task.

[0124] In detail, in an embodiment, the transfer module (TFM) can transform a specific latent vector into a transition vector by mapping it to the latent space of another task based on Riemannian geometry.

[0125] In this process, the transfer module (TFM) can implement geometric alignment between each mapped task according to an embodiment of the present invention. A detailed description of this will be provided later in the multitasking model learning method.

[0126] That is, in the embodiment, the transfer module (TFM) can effectively perform the transfer of knowledge data between multiple tasks by mapping the latent vector according to the first task to the latent space according to the second task through geometric alignment according to the embodiment of the present invention.

[0127] At this time, in the embodiment, the transfer module (TFM) can support data processing that improves the accuracy and consistency of the transformed vector (i.e., the transition vector) by utilizing an autoencoder structure.

[0128] Additionally, in the embodiment, the transfer module (TFM) may include a plurality of transfer modules (TFMs) corresponding to each of the plurality of domains.

[0129] In an embodiment, the transfer module (TFM) may include a first transfer module (TFM) corresponding to a first domain (e.g., boiling point) and a second transfer module (TFM) corresponding to a second domain (e.g., melting point).

[0130] At this time, in the embodiment, one of the plurality of transfer modules (TFMs) may be a source transfer module (TFM), which is a transfer module (TFM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0131] In addition, any one of the remaining transfer modules (TFMs) excluding the source transfer module (TFM) may be a target transfer module (TFM) that is a transfer module (TFM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0132] In addition, the inverse module (ITM) according to an embodiment of the present invention may be a module that reconstructs a transition vector mapped and transformed into a latent space of another task by a transfer module (TFM) so that it is mapped back to the original latent space.

[0133] Thus, in the embodiment, the inverse module (ITM) can generate a vector (hereinafter, an inverse vector) that is a reconstructed and transformed vector back to its original state.

[0134] At this time, the inverse module (ITM) in the embodiment can improve the stability of the above-described reconstruction process and the accuracy and consistency of the corresponding transition vector by utilizing the autoencoder structure.

[0135] In an embodiment, such an inverse module (ITM) may include a plurality of inverse modules (ITMs), each corresponding to a plurality of domains.

[0136] In an embodiment, the inverse module (ITM) may include a first inverse module (ITM) corresponding to a first domain (e.g., boiling point) and a second inverse module (ITM) corresponding to a second domain (e.g., melting point).

[0137] At this time, in the embodiment, one of the plurality of inverse modules (ITMs) may be a source inverse module (ITM), which is an inverse module (ITM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0138] In addition, any one of the remaining inverse modules (ITMs) excluding the source inverse module (ITM) may be a target inverse module (ITM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0139] In addition, a perturbation module (PBM) according to an embodiment of the present invention may be a module that generates a plurality of perturbation vectors by applying a predetermined change to a predetermined embedding vector.

[0140] In detail, in an embodiment, the perturbation module (PBM) may be a module that generates a plurality of perturbation vectors (i.e., perturbation points) around a specific embedding vector by applying a change that moves the specific embedding vector in a predetermined direction.

[0141] At this time, the plurality of generated perturbation vectors are designed to maintain a relative distance from the corresponding embedding vector, thereby effectively assisting geometric alignment.

[0142] That is, the above-mentioned perturbation module (PBM) can help align the coordinate systems between the source task and the target task by generating multiple perturbation vectors to assist in the geometric alignment of the model.

[0143] Additionally, in the embodiment, the perturbation module (PBM) can calculate the distance between a predetermined embedding vector and a plurality of perturbation vectors generated based on the embedding vector, and support matching the displacement between the source task and the target task based on the calculated distance.

[0144] This allows the perturbation module (PBM) to more easily maintain consistency in the latent space for the model.

[0145] According to an embodiment, the perturbation module (PBM) can prevent overfitting of a model and improve generalization performance by forcing a relationship between a given embedding vector and multiple perturbation vectors generated based on the embedding vector to be maintained.

[0146] In addition, the loss calculation module (LCM) according to an embodiment of the present invention may be a module that calculates various loss functions based on various vectors obtained through a multi-task learning model (MtLM).

[0147] In an embodiment, the loss calculation module (LCM) may calculate regression loss, autoencoder loss, consistency loss, mapping loss, distance loss, and / or integrated loss according to an embodiment of the present invention. A detailed description thereof will be provided later in the multi-task model learning method.

[0148] This allows the Loss Calculation Module (LCM) to support regularization and learning for different parts of the model, and to provide feedback for model learning, enabling model optimization.

[0149] Meanwhile, in an embodiment of the present invention, the multi-tasking learning model (MtLM) can perform model optimization and update through various data processing processes linked with the modules described above.

[0150] For example, the multi-task learning model (MtLM) can perform model optimization and parameter update in conjunction with the modules described above based on the AdamW optimization algorithm, etc.

[0151] In this way, in the embodiment of the present invention, the multi-tasking learning model (MtLM) not only simultaneously learns knowledge data according to various domains, but also efficiently learns relationships between multiple domains, thereby expanding the learning area and simultaneously performing effective multi-tasking learning that implements batch learning of local patterns according to each domain and common principles between multiple domains.

[0152] Accordingly, the multi-tasking learning model (MtLM) can directly improve the processing performance and accuracy of various multi-tasking tasks based on the model learned as above.

[0153]

[0154] [Implementation method for providing a multitasking learning model service]

[0155] Hereinafter, a method for implementing a multi-task learning model (MtLM) service that performs multi-tasking based on the geometric alignment of knowledge data of each task's latent space in a single integrated latent space to process multi-tasks for output according to multiple domains by a computing system (1000) according to one embodiment of the present invention is described in detail.

[0156] In general, existing transfer learning techniques are mainly focused on classification tasks of image and / or language data sets, and have limitations in solving regression problems or problems in non-Euclidean spaces.

[0157] In particular, when the training data set is insufficient, the decline in prediction performance for the above-described problem is more inevitable, and when multitasking considering various task types is required, the decline in performance is aggravated in learning and prediction for this.

[0158] Additionally, most existing methods are optimized for handling data in Euclidean space, so they do not work effectively in complex curved spaces or nonlinear spaces.

[0159] FIG. 7 illustrates an example of a conceptual diagram for explaining a multi-tasking model learning method according to one embodiment of the present invention.

[0160] Therefore, as shown in FIG. 7, a computing system (1000) according to one embodiment of the present invention aims to provide a new multi-tasking model learning method that can overcome the regression problem of a small data set and the limitations of existing transfer learning techniques, and a multi-tasking execution method using a machine learning model learned based on the method.

[0161] Hereinafter, in the description according to one embodiment of the present invention, for the sake of effective description, the material described above is limited to a ‘molecule’ and the domain thereof is described based on ‘properties’.

[0162] This is because molecular data sets typically have small amounts of data, contain diverse task types, and primarily deal with regression problems.

[0163] That is, in the case of molecular data sets, various task processing linked to numerous properties is required, but the data provided for this is very limited, and each property has the characteristic of being closely related to or influencing each other.

[0164] Considering these points, the molecular data set is advantageously applicable to multi-task processing across multiple domains, and may be a desirable example for explaining a multi-tasking model learning method according to one embodiment of the present invention and a multi-tasking performance method using a machine learning model learned based on the same.

[0165] However, it is not limited thereto, and it is obvious that any embodiment that can apply multi-tasks according to multiple domains can be included in the embodiments of the present invention.

[0166] Hereinafter, a multitasking model learning method according to one embodiment of the present invention and a multitasking performance method using a machine learning model learned based on the same will be described in more detail with reference to the attached drawings.

[0167] FIG. 8 illustrates a block flow diagram for explaining a multi-tasking model learning method according to one embodiment of the present invention.

[0168] Referring to FIG. 8, a multi-tasking model learning method according to one embodiment of the present invention and a multi-tasking performance method using a machine learning model learned based thereon may include a step (S101) of initializing a multi-tasking learning model (MtLM), a step (S103) of acquiring experimental data, a step (S105) of training the multi-tasking learning model (MtLM) based on the acquired experimental data, and a step (S107) of providing the trained multi-tasking learning model (MtLM).

[0169] In detail, a computing system (1000) according to one embodiment of the present invention can initialize a multi-tasking learning model (MtLM). (S101)

[0170] Here, in other words, the multi-task learning model (MtLM) (Geometrically Aligned Transfer Encoder Model) according to an embodiment of the present invention may be a machine learning model that mutually aligns fragmented knowledge data (for example, latent vectors, etc.) in a latent space for each task through geometric transfer in a single unified latent space (M) in order to process multiple tasks for output according to multiple domains.

[0171] In other words, the multi-tasking learning model (MtLM) according to the embodiment not only simultaneously learns knowledge data according to various domains but also efficiently learns relationships between multiple domains, thereby expanding the learning area and simultaneously performing effective multi-tasking learning that implements batch learning of local patterns according to each domain and common principles between multiple domains.

[0172] In detail, in an embodiment, the computing system (1000) can perform initialization for each component included in the multi-tasking learning model (MtLM) as described above.

[0173] As an example, a computing system (1000) includes an embedding network (MtLM) within a multi-task learning model ( ), encoder network ( ), Regresor (head) network ( ), Transfer Network ( ) and / or inverse network( ) etc. random parameters ( ) can be initialized.

[0174] Additionally, as an example, the computing system (1000) can set a predetermined optimization algorithm to be applied to a multi-tasking learning model (MtLM).

[0175] For example, the computing system (1000) may set the AdamW (Decoupled Weight Decay Regularization) algorithm as the optimization algorithm, and according to an embodiment, the optimization algorithm may be improved and used to independently process weight decay.

[0176] Additionally, the computing system (1000) according to one embodiment of the present invention can obtain experimental data. (S103)

[0177] Here, again, experimental data according to an embodiment of the present invention ( ) may be data including specific material-specific characteristic information and material property-specific information as learning data used for training a multi-task learning model (MtLM).

[0178] At this time, the material characteristic information according to the embodiment may be information specifying the unique characteristics possessed by a given material. That is, the material characteristic information in the embodiment may be information specifying the unique characteristics possessed by a given molecule.

[0179] For example, material characteristic information may include a given material name, molecular structure, and / or chemical formula.

[0180] In addition, the material property specific information according to the embodiment may be information that specifies the data value that a given material has for a given material property.

[0181] For example, material property specific information may include property (i.e., domain) values ​​such as boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity of a given material.

[0182] In detail, in an embodiment, the computing system (1000) can obtain experimental data as described above based on predetermined user input and / or connection with an external server.

[0183] In addition, a computing system (1000) according to one embodiment of the present invention can train a multi-task learning model (MtLM) based on acquired experimental data. (S105)

[0184] FIG. 9 illustrates a block flow diagram for explaining a multi-tasking learning model (MtLM) training method according to one embodiment of the present invention, and FIG. 10 illustrates an example of a conceptual diagram for explaining a multi-tasking learning model (MtLM) training method according to one embodiment of the present invention.

[0185] That is, referring to FIGS. 9 and 10, in the embodiment, the computing system (1000) can perform pre-learning for a multi-task learning model (MtLM) based on the experimental data acquired as described above.

[0186] In detail, in an embodiment, the computing system (1000) can set a training loop for a multi-task learning model (MtLM). (S201)

[0187] In more detail, in an embodiment, the computing system (1000) can set the number of epoch repetitions, the number of task repetitions, and / or the number of batch repetitions during training.

[0188] As an example, the computing system (1000) can set a training loop to repeatedly perform epoch 'i' from '1 to n (n>=1)' during training, repeatedly perform the same for each task 't', and repeatedly perform the same for each preset batch 'b'.

[0189] Additionally, in the embodiment, the computing system (1000) can obtain a geometric alignment vector based on the experimental data obtained as described above. (S203)

[0190] Here, the geometric alignment vector according to the embodiment of the present invention may mean various vectors obtained through a multi-task learning model (MtLM).

[0191] In the embodiment, the geometric alignment vector is an embedding vector ( ), pulturation vector ( ), encoding vectors, transition vectors, and inverse vectors.

[0192] In detail, in an embodiment, the computing system (1000) can input the acquired experimental data into a multi-task learning model (MtLM).

[0193] Additionally, in the embodiment, the computing system (1000) can obtain an embedding vector based on a multi-task learning model (MtLM) that inputs experimental data.

[0194] In more detail, the computing system (1000) can convert the input experimental data into an embedding vector through an embedding network in conjunction with the embedding module (EBM) of the multi-task learning model (MtLM).

[0195] Accordingly, the computing system (1000) can obtain an embedding vector converted into a vector format by projecting the experimental data into a predetermined embedding space.

[0196] Additionally, in the embodiment, the computing system (1000) can generate a perturbation vector based on the acquired embedding vector.

[0197] In detail, in an embodiment, the computing system (1000) can generate a plurality of perturbation vectors (i.e., perturbation points) on a predetermined periphery based on the obtained embedding vector in conjunction with the perturbation module (PBM) of the multi-task learning model (MtLM).

[0198] At this time, in the embodiment, the computing system (1000) can obtain a corresponding perturbation vector for each task by repeatedly performing the above-described functional operation for each task.

[0199] As an example, the computing system (1000) can obtain a fusion vector corresponding to task 't' and a fusion vector corresponding to task 's'.

[0200] In addition, in the embodiment, the computing system (1000) can obtain an encoding vector based on the generated perturbation vector and embedding vector.

[0201] Here, the encoding vector according to the embodiment may include a pultering latent vector, which is a latent vector generated based on a predetermined pultering vector, and an original latent vector generated based on an embedding vector, which is an original vector of the pultering vector.

[0202] In detail, in an embodiment, the computing system (1000) can convert the generated perturbation vector into a latent vector by projecting it into a latent space corresponding to the task through an encoder network in conjunction with the encoder module of a multi-task learning model (MtLM).

[0203] Additionally, in the embodiment, the computing system (1000) can convert the obtained embedding vector into a latent vector by projecting it into a latent space corresponding to the task through an encoder network in conjunction with the encoder module of the multi-task learning model (MtLM).

[0204] Thus, in the embodiment, the computing system (1000) can obtain a perturbation potential vector and an original potential vector.

[0205] At this time, in the embodiment, the computing system (1000) can repeatedly perform the above-described functional operation for each task to obtain the corresponding original potential vector and the perturbation potential vector for each task.

[0206] As an example, the computing system (1000) generates an original latent vector ( : Hereinafter, the original latent vector of the task 't' and the pulverization latent vector corresponding to the task 't' ( : Below, the jet pulverization potential vector can be obtained.

[0207] Additionally, the computing system (1000) generates an original latent vector ( : Below, the original latent vector of the task 's' and the pulverization latent vector corresponding to the task 's' ( : Below, you can obtain the potential vector of the s-pulteration.

[0208] Additionally, in the embodiment, the computing system (1000) can obtain a transition vector based on the obtained encoding vector.

[0209] Here, the transition vector according to the embodiment may include a pulverization transfer vector, which is a transition vector generated based on a predetermined pulverization potential vector, and an original transfer vector, which is a transition vector generated based on an original potential vector corresponding to the pulverization potential vector.

[0210] In detail, in an embodiment, the computing system (1000) can convert the acquired perturbation latent vector and original latent vector into a transition vector by mapping them to the latent space of another task (in an embodiment, task 's' or task 't') through a transfer network in conjunction with a transfer module (TFM) of a multi-task learning model (MtLM).

[0211] Thus, the computing system (1000) can obtain a perturbation transfer vector and an original transfer vector.

[0212] At this time, in the embodiment, the computing system (1000) can repeatedly perform the above-described functional operation for each task to obtain the corresponding original transfer vector and perturbation transfer vector for each task.

[0213] As an example, a computing system (1000) may be configured to provide an original transfer vector ( : Below, the original transfer vector t) and the perturbation transfer vector corresponding to the task 't' ( : Below, the jet pulverization transfer vector can be obtained.

[0214] Additionally, the computing system (1000) provides an original transfer vector ( : Below, the original transfer vector of s) and the transformation transfer vector corresponding to task 's' ( : Below, you can obtain the s perturbation transfer vector.

[0215] Thus, in the embodiment, the computing system (1000) can obtain geometric alignment vectors (i.e., embedding vectors, perturbation vectors, encoding vectors (including original latent vectors and perturbation latent vectors), and transfer vectors (including original transfer vectors and perturbation transfer vectors)) based on experimental data.

[0216] Additionally, in the embodiment, the computing system (1000) can obtain an inverse vector based on the acquired transition vector.

[0217] Here, the inverse vector according to the embodiment may include a pulter inverse vector, which is an inverse vector generated based on a predetermined pulter transfer vector, and an original inverse vector, which is an inverse vector generated based on an original transfer vector corresponding to the pulter transfer vector.

[0218] In detail, in an embodiment, the computing system (1000) can reconstruct the acquired perturbation transfer vector and the original transfer vector through an inverse network so that they are mapped back to the original latent space and converted into inverse vectors in conjunction with the inverse module (ITM) of the multi-task learning model (MtLM).

[0219] Thus, the computing system (1000) can obtain the perturbation inverse vector and the original inverse vector.

[0220] At this time, in the embodiment, the computing system (1000) can repeatedly perform the above-described functional operation for each task to obtain the corresponding original inverse vector and perturbation inverse vector for each task.

[0221] As an example, the computing system (1000) generates an original inverse vector ( : Below, the original inverse vector of the t and the perturbation inverse vector corresponding to the task 't' ( : Below, the jet t perturbation inverse vector can be obtained.

[0222] Additionally, the computing system (1000) generates the original inverse vector ( : Below, the original inverse vector of the task 's' and the perturbation inverse vector corresponding to the task 's' ( : Below, you can obtain the s-pulteration inverse vector.

[0223] Thus, in the embodiment, the computing system (1000) can obtain geometric alignment vectors based on experimental data (i.e., embedding vectors, perturbation vectors, encoding vectors (including original latent vectors and perturbation latent vectors), transition vectors (including original transfer vectors and perturbation transfer vectors), and inverse vectors (including original inverse vectors and perturbation inverse vectors), etc.).

[0224] Additionally, in the embodiment, the computing system (1000) can calculate a geometric alignment loss based on the obtained geometric alignment vector. (S205)

[0225] Here, the geometric alignment loss according to the embodiment of the present invention may mean various loss functions (Loss) calculated based on various vectors (i.e., geometric alignment vectors) obtained through a multi-task learning model (MtLM).

[0226] In the embodiment, the geometric alignment loss is the regression loss ( : Regression loss), autoencoder loss ( : Autoencoder loss), consistency loss( : Consistency loss), mapping loss( : Mapping loss), distance loss( : Distance loss) and / or integration loss( : Integrated loss) etc.

[0227] In the following description, for the sake of effective explanation, the geometric alignment loss is calculated based on the task 't'.

[0228] FIG. 11 and FIG. 12 illustrate examples of drawings for explaining a method for calculating regression loss according to one embodiment of the present invention.

[0229] In detail, referring to FIGS. 10 to 12, in an embodiment, a computing system (1000) can calculate a regression loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0230] In more detail, in the embodiment, the computing system (1000) predicts a predicted value () through a regression module (RGM) according to the following [Mathematical Formula 1]. ) and actual value( , i.e., the regression loss based on the label value) can be calculated. Here, the predicted value of [Mathematical Formula 1] is ' ' can also be expressed as.

[0231] [Mathematical Formula 1]

[0232]

[0233] That is, the computing system (1000) can calculate the regression loss by calculating the mean squared error (MSE) between the predicted value and the actual value.

[0234] At this time, in the embodiment, each task can prevent mutual interference by calculating an independent regression loss based on an encoder module (ECM) and a regressor module (RGM) matching each task and performing learning based on the regression loss.

[0235] In this way, the computing system (1000) can easily evaluate the regression performance of the model by calculating the regression loss.

[0236] In addition, referring further to FIG. 10, in the embodiment, the computing system (1000) can calculate the autoencoder loss2) based on a multi-task learning model (MtLM) that has obtained a geometric alignment vector.

[0237] In detail, in an embodiment, the computing system (1000) can calculate an autoencoder loss based on the original latent vector and the original inverse vector according to the following [Mathematical Formula 2].

[0238] [Mathematical Formula 2]

[0239]

[0240] That is, the computing system (1000) can calculate the autoencoder loss by calculating the mean square error (MSE) between the latent vector and the inverse vector.

[0241] In an embodiment, the computing system (1000) can improve accuracy in the data transfer process through the autoencoder loss calculated as above.

[0242] FIG. 13 illustrates an example of a drawing for explaining an integrated latent space (M) mapping method according to one embodiment of the present invention.

[0243] Meanwhile, referring to FIG. 13, in an embodiment, the computing system (1000) can learn a bidirectional transformation matrix (TM) that can be mapped to a common integrated latent space (M) for each task.

[0244] In detail, in an embodiment, the computing system (1000) can connect latent spaces between tasks by utilizing knowledge data that contain labels for both tasks.

[0245] In this process, the computing system (1000) can calculate consistency loss and mapping loss according to the embodiment.

[0246] FIG. 14 and FIG. 15 illustrate examples of drawings for explaining a consistency loss calculation method according to one embodiment of the present invention.

[0247] In more detail, referring to FIGS. 10, 14 and 15, in an embodiment, a computing system (1000) can calculate consistency loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0248] Specifically, in the embodiment, the computing system (1000) can calculate the consistency loss based on the perturbation transfer vector of task 't' and the perturbation transfer vector of task 's' according to [Mathematical Formula 3] below.

[0249] [Equation 3]

[0250]

[0251] That is, the computing system (1000) can calculate the consistency loss by calculating the mean square error (MSE) between the t-pulter transfer vector and the s-pulter transfer vector.

[0252] At this time, in the embodiment, the computing system (1000) can derive a metric for calculating a distance in space from a transformation matrix (TM), and learn to make the distance in the latent space of each task the same based on the derived metric.

[0253] Through this, the computing system (1000) can more effectively implement geometric alignment between tasks.

[0254] FIG. 16 and FIG. 17 illustrate examples of drawings for explaining a mapping loss calculation method according to one embodiment of the present invention.

[0255] In addition, referring to FIGS. 10, 16 and 17, in the embodiment, the computing system (1000) can calculate a mapping loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0256] In detail, in the embodiment, the computing system (1000) can calculate a mapping loss based on a predicted value based on an actual value according to task 't' and an original inverse vector according to task 's' according to [Mathematical Formula 4] below.

[0257] [Equation 4]

[0258]

[0259] That is, the computing system (1000) can calculate the mapping loss by calculating the mean square error (MSE) between the actual value of task 't' and the predicted value according to the original inverse vector of task 's'.

[0260] In an embodiment, the computing system (1000) can implement learning to transfer latent vectors from the latent space of one task to the latent space of the other task by calculating the mapping loss as described above, and perform the other task based on the transferred vectors, thereby inducing latent characteristics to become similar to each other.

[0261] Through this, the computing system (1000) can evaluate the prediction performance of vectors transferred to the latent space of other tasks and induce learning in a direction to improve it.

[0262] In addition, referring further to FIG. 10, in the embodiment, the computing system (1000) can calculate distance loss5) based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0263] In detail, in the embodiment, the computing system (1000) calculates the distance between the original transfer vector and the perturbation transfer vector of each task according to [Mathematical Equation 5] and [Mathematical Equation 6] below. : The distance loss between tasks can be calculated based on the transfer vector displacement.

[0264] In more detail, in the embodiment, the computing system (1000) calculates the distance between the t original transfer vector and the t perturbation transfer vector according to the task 't' according to [(a) of the mathematical expression 5] below. : Below, the t transfer vector displacement) can be calculated.

[0265] In addition, the computing system (1000) calculates the distance between the s original transfer vector and the s perturbation transfer vector according to the task 's' according to [(b) of the mathematical expression 5] below. : Below, the transfer vector displacement) can be calculated.

[0266] [Equation 5]

[0267]

[0268] Additionally, in the embodiment, the computing system (1000) can calculate the mean square error (MSE) between the t transfer vector displacement and the s transfer vector displacement according to the following [Mathematical Formula 6] to calculate the distance loss.

[0269] [Equation 6]

[0270]

[0271] Here, 'M' in [Mathematical Formula 6] means the number of pulverization points.

[0272] At this time, in the embodiment, the computing system (1000) can define the t transfer vector displacement and the s transfer vector displacement as displacements in the source task and the target task, respectively.

[0273] Thus, the computing system (1000) can more easily calculate the distance between the original transfer vector and the perturbation transfer vector by interpreting the t transfer vector displacement and the s transfer vector displacement as being in a flat Euclidean space.

[0274] Therefore, the computing system (1000) can support more complete consistency maintenance of the latent space of the model.

[0275] FIG. 18 illustrates an example of a drawing for explaining an integrated loss calculation method according to one embodiment of the present invention.

[0276] In addition, referring to FIGS. 10 and 18, in the embodiment, the computing system (1000) can calculate an integrated loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0277] In detail, in the embodiment, the computing system (1000) can calculate an integrated loss by weighting the regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss described above according to [Mathematical Formula 7] below.

[0278] [Equation 7]

[0279]

[0280] At this time, in the embodiment, the computing system (1000) can apply weights to each loss function so that each loss function can be optimized for a specific aspect of the model.

[0281] Here, ' in [Equation 7] ' is the weight of the autoencoder loss, and ' ' is the weight of consistency loss, and ' ' is the weight of the mapping loss, and ' ' is the weight of the distance loss.

[0282] In an embodiment, the computing system (1000) can update parameters in a direction to minimize the integrated loss by adjusting the importance of the loss function corresponding to each weight during the learning process of the model by utilizing the above weights.

[0283] Returning to Figure 9, the computing system (1000) in the embodiment can also perform model optimization and parameter update based on the geometric alignment loss calculated as described above. (S207)

[0284] In detail, in an embodiment, the computing system (1000) can perform optimization and parameter update for a multi-task learning model (MtLM) based on the integrated loss described above.

[0285] As an example, the computing system (1000) can calculate a gradient based on the integrated loss for each parameter of a multi-task learning model (MtLM) through backpropagation.

[0286] And the computing system (1000) can perform parameter update of a multi-task learning model (MtLM) using a calculated gradient and a preset optimization algorithm (e.g., AdamW (Decoupled Weight Decay Regularization) algorithm, etc.).

[0287] Thus, the computing system (1000) can implement a multi-task learning model (MtLM) optimization based on geometric alignment loss (particularly, integration loss).

[0288] In this way, in the embodiment, the computing system (1000) can perform multi-task learning model (MtLM) optimization and parameter update learning through a combination of multiple loss functions calculated in various ways.

[0289] At this time, each loss function can easily assist in improving the performance of the model by correcting the accuracy, consistency, and / or distance of the knowledge data mapping.

[0290] Through this, the computing system (1000) can implement a multi-tasking model that provides improved performance that overcomes the regression problem of a small data set and the limitations of existing transfer learning techniques, while operating more stably and providing improved generalization performance.

[0291] Additionally, in an embodiment, the computing system (1000) may terminate multi-task learning model (MtLM) training. (S209)

[0292] In detail, in an embodiment, the computing system (1000) can terminate the multi-task learning model (MtLM) training process described above when a preset training termination condition is met.

[0293] In an embodiment, the computing system (1000) can terminate multi-task learning model (MtLM) training upon completion of a set training loop.

[0294] Returning to Figure 8, a computing system (1000) according to an embodiment of the present invention can also provide a trained multi-task learning model (MtLM). (S107)

[0295] That is, in the embodiment, the computing system (1000) can provide a multi-task learning model (MtLM) trained as described above in a predetermined manner.

[0296] In an embodiment, the computing system (1000) may provide a multi-task learning model (MtLM) trained according to an embodiment of the present invention in conjunction with a predetermined application service (e.g., a material synthesis / evaluation service, a material property prediction service, and / or an optimal material recommendation service, etc.).

[0297] Thus, the computing system (1000) can effectively support processing of various multitasking tasks using a multitasking learning model (MtLM) with improved performance.

[0298] In this way, in the embodiment, the computing system (1000) can provide a multi-task learning model (MtLM) that provides improved performance that overcomes the regression problem of a small data set and the limitations of existing transfer learning techniques by mutually transferring and learning knowledge data of each task's latent space through geometric alignment in a single integrated latent space in order to process multiple tasks for output according to multiple domains, while operating more stably.

[0299] Through this, the computing system (1000) can provide a transfer learning-based multi-tasking model that operates stably and robustly with high generalization performance even in situations where the amount of given data is small, various task types are included, or regression problems are mainly dealt with.

[0300] In other words, the computing system (1000) can provide a multi-task learning model (MtLM) with improved prediction performance based on knowledge distilled through geometric alignment-based transfer learning performed in conjunction with other domains, even if there is a domain among multiple domains (in the embodiment, physical properties) that lacks experimental data (learning data).

[0301] For example, the computing system (1000) pre-trains a multi-task learning model (MtLM) based on the first to tenth properties for each of a plurality of molecular structural formulas, and then, when a first molecular structural formula including only data for the first to fifth properties is input, the computing system (1000) can more accurately predict data values ​​for the remaining sixth to tenth properties for the first molecular structural formula based on the knowledge data transferred and distilled through pre-training, and generate and provide output data based thereon.

[0302] In this way, the computing system (1000) according to an embodiment of the present invention can provide a multi-tasking model that implements effective transfer learning based on geometric alignment, guarantees high generalization performance, improves prediction accuracy for regression problems, supports regularization according to a combination of various loss functions, and performs a stable learning process to guarantee robust performance.

[0303] Above, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using a machine learning model learned based on the same have the effect of providing a multitasking model that maintains high performance even on a small data set by solving the problem of insufficient data by transferring knowledge learned from a source task to a target task through transfer learning.

[0304] Therefore, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using a machine learning model learned based on the same have the effect of expanding the scope of application to fields where it was difficult to apply the machine learning model due to insufficient data or domain knowledge.

[0305] In addition, the multi-tasking model learning method according to one embodiment of the present invention and the multi-tasking performance method using a machine learning model learned based thereon provide a specialized transfer learning technique that can be effectively applied to regression problems, thereby having the effect of demonstrating high prediction performance even in complex regression problems such as molecular data sets.

[0306] In addition, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using the machine learning model learned based thereon have the effect of improving the efficiency of transfer learning by maintaining geometric consistency between tasks by optimizing knowledge transfer between source tasks and target tasks through a Riemannian geometric approach.

[0307] In addition, the multi-tasking model learning method according to one embodiment of the present invention and the multi-tasking performance method using a machine learning model learned based thereon have the effect of further improving the generalization performance of the model by combining multiple loss functions to normalize various aspects of the model.

[0308] Therefore, the multitasking model learning method according to one embodiment of the present invention and the multitasking performance method using the machine learning model learned based thereon provide a multitasking model that can be universally utilized for various substances (materials), thereby having the effect of improving the quality of the related industry as a whole.

[0309]

[0310] Meanwhile, the embodiments of the present invention described above may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. Hardware devices may be changed into one or more software modules to perform processing according to the present invention, and vice versa.

[0311] The specific implementations described in the present invention are exemplary embodiments and do not limit the scope of the present invention in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the lines connecting or connecting members between components illustrated in the drawings are merely representative of functional connections and / or physical or circuit connections, and may be replaced or represented as various additional functional connections, physical connections, or circuit connections in an actual device. In addition, unless specifically mentioned as “essential,” “important,” etc., a component may not be absolutely necessary for the application of the present invention.

[0312] Although the detailed description of the present invention has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art or having ordinary knowledge in the art that various modifications and changes can be made to the present invention without departing from the spirit and technical scope of the present invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

[0313] The present invention relates to a multitasking model learning method and a multitasking performance method using a machine learning model learned based on the same, and is applicable to the artificial intelligence industry, and thus has industrial applicability.

Claims

1. A method for training a multitasking model in a computing system including memory and processor, A step of initializing a multi-tasking model that handles multiple tasks based on multiple domains; A step of acquiring certain experimental data; and A step of training the multi-tasking model based on the acquired experimental data is included, The steps for training the above multi-tasking model are: A step of obtaining a geometric alignment vector, which is a vector that supports geometric alignment between data in a single integrated latent space (Manifold) based on the above experimental data, A step of calculating geometric alignment loss based on the geometric alignment vector obtained above, A step of updating the parameters of the multi-tasking model based on the geometric alignment loss produced above. How to train a multitasking model.

2. In paragraph 1, The step of obtaining the above geometric alignment vector is: A step of obtaining an embedding vector converted into a vector format by projecting the experimental data into a predetermined embedding space based on the embedding module included in the above multi-tasking model. How to train a multitasking model.

3. In paragraph 2, The step of obtaining the above geometric alignment vector is: A step of obtaining a plurality of perturbation vectors by moving the embedding vector in a predetermined direction based on the perturbation module included in the multitasking model is further included. How to train a multitasking model.

4. In paragraph 3, The step of obtaining the above geometric alignment vector is: A step of obtaining an original latent vector by projecting the embedding vector onto the latent space of the first task (Task 1) based on the encoder module included in the multi-task model; A step of obtaining a perturbation latent vector by projecting the perturbation vector onto the latent space of the first task based on the encoder module included in the multitasking model is further included. How to train a multitasking model.

5. In paragraph 4, The step of obtaining the above geometric alignment vector is: A step of obtaining an original transfer vector by mapping the original latent vector to the latent space of the second task (Task 2) based on the transfer module included in the above multi-tasking model, A method further comprising: obtaining a perturbation transfer vector by mapping the perturbation latent vector to the latent space of a second task based on the transfer module included in the multitasking model; How to train a multitasking model.

6. In paragraph 5, The step of obtaining the above geometric alignment vector is: A step of obtaining an original inverse vector by remapping the original transfer vector to the latent space of the first task based on the inverse transfer module included in the multitasking model; A step of obtaining a perturbation inverse vector by remapping the perturbation transfer vector to the latent space of the first task based on the inverse module included in the multitasking model is further included. How to train a multitasking model.

7. In paragraph 6, The step of calculating the above geometric alignment loss is: A step of calculating regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss based on the above geometric alignment vector, A step of calculating an integrated loss by weighting the above-mentioned calculated regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss. How to train a multitasking model.

8. In paragraph 7, The step of updating the parameters of the above multitasking model is: including a step of updating the parameters in a direction that minimizes the above integration loss. How to train a multitasking model.

9. In the 7th paragraph, The step of calculating the above geometric alignment loss is: A step of obtaining a prediction value according to the latent vector based on the regressor module included in the above multi-tasking model, A step of calculating the mean squared error based on the obtained predicted value and the label value corresponding to the latent vector, Further comprising a step of calculating the regression loss based on the calculated mean square error. How to train a multitasking model.

10. In Article 7, The step of calculating the above geometric alignment loss is: A step of calculating the mean square error based on the original latent vector and the original inverse vector, Further comprising a step of calculating the autoencoder loss based on the calculated mean square error. How to train a multitasking model.

11. In Article 7, The step of calculating the above geometric alignment loss is: A step of calculating a mean square error based on a perturbation transfer vector mapped from the latent space of the first task to the latent space of the second task and a perturbation transfer vector mapped from the latent space of the second task to the latent space of the first task; Further comprising a step of calculating the consistency loss based on the calculated mean square error. How to train a multitasking model.

12. In Article 7, The step of calculating the above geometric alignment loss is: A step of calculating the mean square error based on the label value based on the first task and the predicted value based on the original inverse vector based on the second task, Further comprising a step of calculating the mapping loss based on the calculated mean square error. How to train a multitasking model.

13. In Article 7, The step of calculating the above geometric alignment loss is: A step of calculating a first transfer vector displacement, which is the distance between the original transfer vector and the perturbation transfer vector based on the first task, A step of calculating a second transfer vector displacement, which is the distance between the original transfer vector and the perturbation transfer vector based on the second task, A step of calculating a mean square error based on the calculated first transfer vector displacement and the second transfer vector displacement, Further comprising a step of calculating the distance loss based on the calculated mean square error. How to train a multitasking model.

14. In paragraph 1, The above experimental data is, It includes material-specific characteristic information, which is information that specifies the unique characteristics possessed by a given material, and material property-specific information, which is information that specifies the data value that a given material has for a given physical property. How to train a multitasking model.

15. At least one memory; and At least one processor for training a multitasking model by reading at least one application stored in the memory; The instructions of the above processor are: A step for initializing a multi-tasking model that handles multiple tasks based on multiple domains, A step of acquiring certain experimental data, A step of obtaining a geometric alignment vector, which is a vector that supports geometric alignment between data in a single integrated latent space (Manifold) based on the experimental data obtained above, A step of calculating geometric alignment loss based on the geometric alignment vector obtained above, A command that includes a step of updating the parameters of the multi-tasking model based on the geometric alignment loss produced above. Multitasking model training server.

Images