A method for learning a multitasking model and a method for executing multitasking using a machine learning model learned therefrom.

Abstract

The multitasking model learning method and the multitasking execution method using a machine learning model learned thereon, according to an embodiment of the present invention, provide a multitasking model learning method and a multitasking execution method using a machine learning model learned thereon, which involve transferring and learning knowledge data from the latent spaces of each task via geometric alignment in a single integrated latent space in order to process multiple tasks for output from multiple domains.

Description

Technical Field

[0001] The present invention relates to a multi-tasking model learning method and a multi-tasking execution method using a machine learning model learned based on the same. More specifically, in order to process multiple tasks for output by a plurality of domains, knowledge data of a latent space for each task is mutually transferred and learned through geometric alignment in one integrated latent space, and a multi-tasking model learning method and a multi-tasking execution method using a machine learning model learned based on the same.

Background Art

[0002] Machine learning and artificial intelligence models require large amounts of data. However, in reality, there are limitations to always having sufficient data. This is further exacerbated especially when trying to apply the model to a new domain or task. Typically, the molecular structure dataset is a good example of such a situation. In the fields of chemistry and pharmacy, data is required to predict the properties of new molecules, but experimental data for each molecule is difficult to obtain and requires a lot of cost. Therefore, the need for a transfer learning technique that applies the knowledge of an already learned model to a new task has increased.

[0003] However, existing transfer learning has mainly developed focusing on classification problems of large-scale datasets such as image and text data. Therefore, existing transfer learning techniques and the like show limitations when applied to regression problems or small-scale, complex datasets such as molecular datasets. In particular, in problems such as molecular structure data that are high-dimensional and where the relationship between each component and bond is very important, existing Euclidean space-based transfer learning techniques and the like cannot effectively process such complex structures in non-Euclidean spaces.

[0004] On the other hand, Riemannian geometry enables calculus in curved space, allowing for better representation and analysis of complex data structures. Such Riemannian geometric approaches assume that latent vectors reside on a curved manifold, which is advantageous for aligning the geometry between the source and target tasks.

[0005] Therefore, based on the above background, it is necessary to introduce new technologies that can achieve high predictive performance and stability even with small datasets, realize more effective transfer learning, and simultaneously improve model normalization performance and its generalization performance. [Overview of the project] [Problems that the invention aims to solve]

[0006] One embodiment of the present invention aims to provide a multitasking model learning method for processing multiple tasks for output from multiple domains, by transferring and learning knowledge data from each task's latent space through geometric alignment in a single integrated latent space, and a multitasking execution method using a machine learning model learned therefrom.

[0007] However, the technical problems that the present invention and its embodiments aim to address are not limited to those described above, and other technical problems may exist. [Means for solving the problem]

[0008] A multitasking model learning method according to an embodiment of the present invention is a method for a computing system comprising memory and a processor to train a multitasking model, comprising the steps of: initializing a multitasking model that processes multiple tasks based on multiple domains; acquiring predetermined experimental data; and training the multitasking model based on the acquired experimental data, wherein the step of training the multitasking model comprises the steps of: acquiring geometric alignment vectors, which are vectors that support geometric alignment between data in a single integrated latent space (Manifold), based on the experimental data; calculating a geometric alignment loss based on the acquired geometric alignment vectors; and updating the parameters of the multitasking model based on the calculated geometric alignment loss.

[0009] In other respects, the step of obtaining the geometric alignment vector includes the step of obtaining an embedding vector by projecting the experimental data onto a predetermined embedding space and converting it into a vector format, based on an embedding module included in the multitasking model.

[0010] In other aspects, the step of obtaining the geometric alignment vector further includes the step of obtaining a plurality of perturbation vectors obtained by moving the embedding vector in a predetermined direction based on a perturbation module included in the multitasking model.

[0011] In other respects, the step of obtaining the geometric alignment vector further includes the step of obtaining an original latent vector by projecting the embedding vector onto the latent space of Task 1 based on an encoder module included in the multitasking model, and the step of obtaining a perturbation latent vector by projecting the perturbation vector onto the latent space of Task 1 based on an encoder module included in the multitasking model.

[0012] In other respects, the step of obtaining the geometric alignment vector further includes the step of obtaining an original transfer vector by mapping the original latent vector to the latent space of a second task (Task 2) based on a transfer module included in the multitasking model, and the step of obtaining a perturbation transfer vector by mapping the perturbation latent vector to the latent space of a second task (Task 2) based on a transfer module included in the multitasking model.

[0013] In other respects, the step of obtaining the geometric alignment vector further includes the step of obtaining an original inverse vector by remapping the original transfer vector to the latent space of the first task based on an inverse transfer module included in the multitasking model, and the step of obtaining a perturbation inverse vector by remapping the perturbation transfer vector to the latent space of the first task based on an inverse module included in the multitasking model.

[0014] In other respects, the step of calculating the geometric alignment loss includes the steps of calculating regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss based on the geometric alignment vector, and calculating an integrated loss which is a weighted sum of the calculated regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss.

[0015] In other respects, the step of updating the parameters of the multitasking model includes the step of updating the parameters in a manner that minimizes the combined loss.

[0016] In other respects, the step of calculating the geometric alignment loss further includes the steps of obtaining predicted values ​​from the latent vector based on the Regressor Module included in the multitasking model, calculating the Mean Squared Error based on the obtained predicted values ​​and the Label values ​​corresponding to the latent vector, and calculating the regression loss based on the calculated Mean Squared Error.

[0017] In other respects, the step of calculating the geometric alignment loss further includes the step of calculating the mean squared error based on the original latent vector and the original inverse vector, and the step of calculating the autoencoder loss based on the calculated mean squared error.

[0018] In other respects, the step of calculating the geometric alignment loss further includes the steps of calculating a mean squared error based on perturbation transfer vectors mapped from the latent space of the first task to the latent space of the second task and perturbation transfer vectors mapped from the latent space of the second task to the latent space of the first task, and calculating the consistency loss based on the calculated mean squared error.

[0019] In other respects, the step of calculating the geometric alignment loss further includes the steps of calculating the mean squared error based on the label values ​​based on the first task and the predicted values ​​based on the original inverse vectors based on the second task, and calculating the mapping loss based on the calculated mean squared error.

[0020] In other respects, the step of calculating the geometric alignment loss further includes the steps of: calculating a first transfer vector displacement which is the distance between the original transfer vector and the perturbation transfer vector based on the first task; calculating a second transfer vector displacement which is the distance between the original transfer vector and the perturbation transfer vector based on the second task; calculating a mean square error based on the calculated first transfer vector displacement and the second transfer vector displacement; and calculating the distance loss based on the calculated mean square error.

[0021] In other respects, the experimental data includes material-specific property information, which is information that identifies the unique properties possessed by a given substance, and material property identification information, which is information that identifies the data values ​​that a given substance possesses for a given physical property.

[0022] On the other hand, a multitasking model learning server according to an embodiment of the present invention comprises at least one memory and at least one processor that reads at least one application stored in the memory and trains a multitasking model, wherein the processor's instruction words include instructions for initializing a multitasking model that processes multiple tasks based on multiple domains, acquiring predetermined experimental data, acquiring a geometric alignment vector which is a vector that supports geometric alignment between data in a single integrated latent space (Manifold) based on the acquired experimental data, calculating a geometric alignment loss based on the acquired geometric alignment vector, and updating the parameters of the multitasking model based on the calculated geometric alignment loss. [Effects of the Invention]

[0023] A multitasking model learning method and a multitasking execution method using a machine learning model learned therefrom, according to one embodiment of the present invention, have the effect of providing a multitasking model that can maintain high performance even with small datasets by transferring knowledge learned in a source task to a target task via transfer learning, thereby resolving the problem of insufficient data.

[0024] Therefore, the multitasking model learning method according to one embodiment of the present invention and the multitasking execution method using the machine learning model learned therefrom have the effect of extending the scope of application to fields where it was difficult to apply machine learning models due to a lack of data and domain knowledge.

[0025] In addition, the multi-tasking model learning method according to an embodiment of the present invention and the multi-tasking execution method using the machine learning model learned based on this provide a specialized transfer learning technique that can be effectively applied to regression problems, and thus have the effect of being able to exhibit high prediction performance even in complex regression problems such as molecular datasets.

[0026] In addition, the multi-tasking model learning method according to an embodiment of the present invention and the multi-tasking execution method using the machine learning model learned based on this optimize the knowledge transfer between the source task and the target task through Riemannian geometric proximity, and thus have the effect of being able to improve the efficiency of transfer learning while maintaining the geometric consistency between tasks.

[0027] In addition, the multi-tasking model learning method according to an embodiment of the present invention and the multi-tasking execution method using the machine learning model learned based on this combine a plurality of loss functions to normalize various aspects of the model, and thus have the effect of being able to further enhance the generalization performance of the model.

[0028] Therefore, the multi-tasking model learning method according to an embodiment of the present invention and the multi-tasking execution method using the machine learning model learned based on this provide a multi-tasking model that can be generally used for various substances (materials), and have the effect of being able to enhance the quality of the entire related industry.

[0029] However, the effects that can be obtained in the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood from the following description.

Brief Description of the Drawings

[0030] [Figure 1] An example of a block diagram of a computing system that realizes a multi-tasking learning model providing service according to an embodiment of the present invention is shown. [Figure 2] An example block diagram of a computing device that implements a multitasking learning model provision service according to one embodiment of the present invention is shown. [Figure 3] This shows an example of a block diagram of another aspect of a computing device that implements a multitasking learning model provisioning service according to one embodiment of the present invention. [Figure 4] and [Figure 5] An example of a conceptual diagram illustrating a multitasking learning model according to one embodiment of the present invention is shown. [Figure 6] This shows an internal block diagram of a multitasking learning model according to one embodiment of the present invention. [Figure 7] An example of a conceptual diagram illustrating a multitasking model learning method according to one embodiment of the present invention is shown. [Figure 8] A flowchart of blocks is shown to illustrate a multitasking model learning method according to one embodiment of the present invention. [Figure 9] A flowchart of blocks is shown to illustrate a multitasking learning model training method according to one embodiment of the present invention. [Figure 10] An example of a conceptual diagram illustrating a multitasking learning model training method according to one embodiment of the present invention is shown. [Figure 11] and [Figure 12] An example diagram illustrating a regression loss calculation method according to one embodiment of the present invention is shown. [Figure 13] An example diagram illustrating an integrated latent space mapping method according to one embodiment of the present invention is shown. [Figure 14] and [Figure 15] An example diagram illustrating a consistency loss calculation method according to one embodiment of the present invention is shown. [Figure 16] and [Figure 17] An example diagram illustrating a mapping loss calculation method according to one embodiment of the present invention is shown. [Figure 18] An example diagram illustrating the integrated loss calculation method according to one embodiment of the present invention is shown. [Modes for carrying out the invention]

[0031] The present invention can be modified in various ways and has various embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, and the methods for achieving them, will become clear when referred to the embodiments described in detail below, along with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be realized in various forms. In the following embodiments, terms such as "first," "second," etc., are used not in a restrictive sense but to distinguish one component from another. Also, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, terms such as "includes" or "has" mean that the features or components described in the specification exist, and do not preclude the possibility of adding one or more other features or components. Also, in the drawings, for illustrative purposes, the size of components, etc., may be exaggerated or reduced. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for illustrative purposes, and the present invention is not necessarily limited to what is illustrated.

[0032] Embodiments of the present invention will be described in detail below with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components will be given the same reference numerals, and redundant descriptions thereof will be omitted.

[0033] [An example system that realizes a multitasking learning model provision service]

[0034] The following describes in detail, with reference to the attached diagrams, an exemplary system that provides a multitasking learning model service for processing multiple tasks for output from multiple domains. This system involves transferring and learning knowledge data from each task's latent space into a single integrated latent space via geometric alignment, and then performing multitasking based on this.

[0035] Figure 1 shows an example block diagram of a computing system that implements a multitasking learning model provision service according to one embodiment of the present invention.

[0036] As shown in Figure 1, the computing system 1000 that realizes the multitasking learning model provision service of the present invention includes a user computing device 110, a server computing system 130, and a training computing system 150, and the devices and the like are able to communicate via a network 170.

[0037] A multitasking model learning method and a multitasking execution method using a machine learning model learned therefrom, according to one embodiment of the present invention, can be implemented and provided locally by a user computing device 110, by a server computing system 130 that communicates with the user computing device 110 in the form of a web service, and by a user computing device 110 and a server computing system 130 that cooperate with each other to implement and provide the method.

[0038] In this embodiment, the user computing device 110 and / or the server computing system 130 can train the machine learning models 120 and / or 140 through interaction with a training computing system 150 which is communicatively connected via a network 170. The training computing system 150 may be separate from the server computing system 130 or may be part of the server computing system 130.

[0039] In this system, the artificial intelligence model can be trained in three ways: 1) by the user computing device 110 directly locally; 2) by the server computing system 130 and the user computing device 110 interacting with each other via the network 170; and 3) by a separate training computing system 150 using various training and learning techniques. Furthermore, the training computing system 150 can transmit and provide / update the trained artificial intelligence model to the user computing device 110 and / or the server computing system 130 via the network 170.

[0040] In some embodiments, the training computing system 150 may be part of the server computing system 130 or part of the user computing device 110.

[0041] The user computing device 110 may include smartphones, mobile phones, digital broadcasting devices, PDAs (personal digital assistants), PMPs (portable multimedia players), desktops, wearable devices, embedded computing devices, and / or tablet PCs, as well as all other types of computing devices.

[0042] Such a user computing device 110 comprises at least one processor 111 and memory 112. Here, the processor 111 can consist of at least one or more electrically connected processors from among a central processing unit (CPU), graphics processing unit (GPU), application-specific integrated circuits (ASICs), digital signal processors (DSSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for functional execution.

[0043] The memory 112 may include one or more non-temporary / temporary computer-readable storage media and combinations thereof, such as RAM, ROM, EEPROM, EPROM, flash memory devices, and magnetic disks, and may include web storage of a server that performs memory storage functions over the internet. Such memory 112 can store data 113 and instruction words 114 necessary for the at least one processor 111 to perform functional operations such as training an artificial intelligence model or performing multitasking learning through the artificial intelligence model.

[0044] In one embodiment, the user computing device 110 can store at least one or more machine learning models 120.

[0045] Specifically, the machine learning model 120 can be various machine learning models such as multiple neural networks (e.g., deep neural networks) or other types of machine learning models including nonlinear and / or linear models, and can be composed of combinations thereof.

[0046] In this case, the neural network may include at least one of the following: feed-forward neural networks, cyclic neural networks (e.g., long-short-term memory cyclic neural networks), convolutional neural networks, and / or other forms of neural networks.

[0047] In one embodiment, the user computing device 110 receives at least one or more machine learning models 120 from the server computing system 130 via the network 170, stores them in the memory 112, and then executes the stored machine learning models 120 using the processor 111 to perform multitasking learning and the like.

[0048] In another embodiment, the server computing system 130 includes at least one machine learning model 140 and operates via the machine learning model 140, and can provide a multitasking learning model provision service to the user by coordinating with the user computing device 110 in a manner that communicates with the user computing device 110 and related data.

[0049] For example, the user computing device 110 can provide a multitasking learning model service in which the server computing system 130 uses the machine learning model 140 via the web to provide output in response to user input.

[0050] Furthermore, the artificial intelligence model can also be realized in a manner in which at least a portion of the machine learning models 120 and / or 140 are executed on the user computing device 110, and the remainder is executed on the server computing system 130.

[0051] Furthermore, the user computing device 110 may include at least one or more input components 121 that sense user input. For example, the user input component 121 may include a touch sensor (e.g., a touchscreen and / or touchpad, etc.) that senses touch from a user's input medium (e.g., a finger or stylus), an image sensor that senses user motion input, a microphone that senses user voice input, buttons, a mouse and / or a keyboard, etc. The user input component 121 may also include an interface and an external controller (e.g., a mouse and / or keyboard, etc.) if it receives input to an external controller via an interface.

[0052] The server computing system 130 comprises at least one processor 131 and memory 132. Here, the processor 131 can consist of at least one or more electrically connected processors from among a central processing unit (CPU), graphics processing unit (GPU), ASICs (application-specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or other electrical units for functional execution.

[0053] The memory 132 may include one or more non-temporary / temporary computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, and magnetic disks, or combinations thereof. Such a memory 132 can store data 133 and instruction words 134 necessary for the processor 131 to perform functional operations, such as training an artificial intelligence model or performing multitasking learning through the artificial intelligence model.

[0054] In one embodiment, the server computing system 130 can be implemented by including at least one computing device. For example, the server computing system 130 can be implemented so that multiple computing devices operate in a sequential computing architecture, a parallel computing architecture, or a combination thereof. The server computing system 130 can also include multiple computing devices connected by a network 170.

[0055] Furthermore, the server computing system 130 can store at least one or more machine learning models 140. For example, the server computing system 130 may include neural networks and / or other multi-layer nonlinear models as machine learning models 140. Exemplary neural networks may include feedforward neural networks, deep neural networks, cyclic neural networks, and convolutional neural networks.

[0056] The training computing system 150 comprises at least one processor 151 and memory 152. Here, the processor 151 can consist of at least one or more electrically connected processors from among a central processing unit (CPU), graphics processing unit (GPU), application-specific integrated circuits (ASICs), digital signal processors (DSSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for functional execution.

[0057] The memory 152 may include one or more non-temporary / temporary computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, and magnetic disks, or combinations thereof. Such a memory 152 can store data 153 and instruction words 154 necessary for the processor 151 to perform tasks such as training an artificial intelligence model.

[0058] For example, the training computing system 150 may include a model trainer 160 that trains machine learning models 120 and / or 140 stored in the user computing device 110 and / or server computing system 130 using various training or learning techniques, such as backward propagation of errors (according to the framework shown in Figure 3).

[0059] For example, such a model trainer 160 can perform backpropagation-based updates to one or more parameters of the machine learning models 120 and / or 140 based on a defined loss function.

[0060] In some implementations, error backpropagation may include truncated backpropagation through time. The model trainer 160 may perform several generalization techniques (e.g., weight reduction, dropout, and / or knowledge distillation) to improve the generalization capabilities of the machine learning models 120 and / or 140 being trained.

[0061] In particular, the model trainer 160 can train the machine learning models 120 and / or 140 based on a series of training data 161. Here, the training data 161 may include data of different forms, such as images, audio samples, and / or text. Examples of image types that can be used may include video frames, LiDAR point clouds, X-ray images, computed tomography scans, sec-second spectral images, and / or various other forms of images.

[0062] Such training data 161 can be provided by the user computing device 110 and / or the server computing system 130. When the training computing device trains the machine learning models 120 and / or 140 on specific data from the user computing device 110, the machine learning models 120 and / or 140 can be characterized into personalized models.

[0063] The model trainer 160 also includes computer logic that is utilized to provide the desired functionality.

[0064] Furthermore, the model trainer 160 can be implemented by hardware, firmware, and / or software that control a general-purpose processor. In one implementation, the model trainer 160 includes a program file stored in a storage device, which is loaded into memory 152 and can be executed by one or more processors 151. In another implementation, the model trainer 160 includes one or more sets of computer-executable data 153 and instruction words 154, etc., stored in a tangible computer-readable storage medium such as a RAM hard disk or an optical or magnetic medium.

[0065] Network 170 includes, but is not limited to, 3GPP (3rd Generation Partnership Project) networks, LTE (Long Term Evolution) networks, WiMAX (World Interoperability for Microwave Access) networks, the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth networks, satellite broadcasting networks, analog broadcasting networks, and / or DMB (Digital Multimedia Broadcasting) networks.

[0066] In general, communication over network 170 can be conducted using any type of wired and / or wireless connection, via various communication protocols (e.g., TCP / IP, HTTP, SMTP, and / or FTP), encodings or formats (e.g., HTML and / or XML), and / or protective skimmers (e.g., VPN, secure HTTP, and / or SSL).

[0067] Figure 2 shows an example block diagram of a computing device that implements a multitasking learning model provision service according to one embodiment of the present invention.

[0068] As shown in Figure 2, the computing device 100 included in the user computing device 110, the server computing system 130, and the training computing system 150 includes multiple applications (e.g., Application 1 to Application N). Each application may include a machine learning library and one or more machine learning models. For example, applications may include image processing applications (e.g., Detection, Classification, and / or Segmentation), text messaging applications, email applications, writing applications, virtual keyboard applications, browser applications, and / or chatbot applications.

[0069] In one embodiment, the computing device 100 may include a model trainer 160 for training an artificial intelligence model, and by storing and operating the trained artificial intelligence model, it can provide output data based on predetermined input data (in one embodiment, material-specific property information and / or material physical property identification information, etc.).

[0070] Each application of the computing device 100 can communicate with several other components of the computing device 100, such as at least one sensor, a context manager, a device state component, and / or additional components. In one embodiment, each application can communicate with each device component using an API (e.g., a public API). In one embodiment, the API used by each application may be specific to that application.

[0071] Figure 3 shows an example of a block diagram of another aspect of a computing device that implements a multitasking learning model provisioning service according to one embodiment of the present invention.

[0072] As shown in Figure 3, the computing device 200 includes multiple applications (e.g., Application 1 to Application N). Each application can communicate with the central intelligence layer. For example, applications may include an image processing application, a text messaging application, an email application, a writing application, a virtual keyboard application, and / or a browser application. In one embodiment, each application can communicate with the central intelligence layer (and the models stored therein) using an API (e.g., a common API across all applications).

[0073] The central intelligence layer can include multiple machine learning models. For example, as shown in Figure 3, at least a portion of each machine learning model can be provided to each application and managed by the central intelligence layer. In other implementations, two or more applications can share a single machine learning model. For example, in some implementations, the central intelligence layer can provide a single model to all applications. In some implementations, the central intelligence layer can be implemented within or separately from the operational structure of computing device 200.

[0074] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized data storage for the computing device 200. As shown in Figure 3, the central device data layer can communicate with several other components of the computing device 200, such as one or more sensors, context managers, device state components, and / or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0075] The technologies described herein refer not only to servers, databases, software applications, and other computer infrastructure systems, but also to actions taken and information transmitted to or from said systems. The inherent flexibility of computer infrastructure systems will be recognized as allowing for a wide range of possible configurations, combinations, and divisions of work, as well as functionality between and from components. For example, the processes described herein can be implemented using a single device, or multiple devices or components operating as a single device or component or combination thereof. Databases and applications can be implemented in a single system or in a distributed system across multiple systems. Distributed components can operate sequentially or in parallel.

[0076] [Multi-tasking Learning Model (MtLM)]

[0077] Figures 4 and 5 show illustrative conceptual diagrams illustrating a multitasking learning model (MtLM) according to one embodiment of the present invention.

[0078] As shown in Figures 4 and 5, the Geometrically Aligned Transfer Encoder Model (MtLM) according to an embodiment of the present invention can be a machine learning model that aligns fragmented knowledge data (such as latent vectors in an embodiment) in a task-specific latent space via geometric transfer in a single integrated latent space (M: Manifold) in order to process multiple tasks for output from multiple domains.

[0079] In other words, the multitasking learning model (MtLM) according to this embodiment not only learns knowledge data from various domains simultaneously, but also efficiently learns various inter-domain relationships, thereby expanding the learning domain and enabling effective multitasking learning that simultaneously learns local patterns from each domain and common principles across multiple domains.

[0080] As a result, the Multitasking Learning Model (MtLM) can directly improve the processing performance and accuracy of various multitasking tasks based on the model learned as described above.

[0081] In this embodiment, such a multitasking learning model (MtLM) can perform pre-training based on predetermined experimental data.

[0082] Here, the experimental data according to the embodiment is training data used to train a multitasking learning model (MtLM), and may include predetermined material-specific property information and material property identification information.

[0083] In this case, the material-specific property information according to the embodiment can be information that identifies the unique properties possessed by a given material.

[0084] For example, material-specific property information may include a predetermined material name, molecular structural formula, and / or chemical formula.

[0085] Furthermore, the material property identification information according to the embodiment can be information that identifies the data values ​​that a given material possesses for a given physical property.

[0086] For example, material property identification information may include property (i.e., domain) values ​​such as boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity of a given substance.

[0087] On the other hand, in the embodiment, a multitasking learning model (MtLM) that has been pre-trained as described above can receive predetermined material-specific characteristic information and / or material property identification information, and output data predicted by the received information and learned knowledge.

[0088] As an embodiment, a multitasking learning model (MtLM) can receive predetermined material-specific property information and output material property identification information predicted by the received information and learned knowledge.

[0089] In another embodiment, a multitasking learning model (MtLM) can receive predetermined material property identification information and output material-specific property information predicted by the received information and learned knowledge.

[0090] In yet another embodiment, the multitasking learning model (MtLM) can receive predetermined material-specific characteristic information and material property identification information, and output the optimal material-specific characteristic information and material property identification information predicted by the received information and learned knowledge.

[0091] Figure 6 shows an internal block diagram of a multitasking learning model (MtLM) according to one embodiment of the present invention.

[0092] As shown in Figure 6, in other aspects, the multitasking learning model (MtLM) according to the embodiment may include at least one embedding module (EBM), an encoder module (ECM), a regressor module (RGM), a transfer module (TFM), an inverse transfer module (ITM), a perturbation module (PBM), and a loss calculation module (LCM).

[0093] Specifically, an embedding module (EBM) according to an embodiment of the present invention can be a pre-encoder module that converts predetermined input data into an embedding vector.

[0094] In other words, an embedding module (EBM) can be a module that projects specific input data onto a predetermined embedding space and converts it into a vector format.

[0095] As an embodiment, the embedding module (EBM) can provide an embedding vector for input data based on a DMPNN (Directed Message Passing Neural Network) structure.

[0096] Furthermore, an encoder module (ECM) according to an embodiment of the present invention can be a module that takes a predetermined embedding vector as input and projects the input embedding vector onto a latent space corresponding to the task to convert it into a latent vector.

[0097] In other words, an encoder module (ECM) can be a module that extracts the main features of an input embedding vector and represents them in the corresponding latent space.

[0098] In an embodiment, such an encoder module (ECM) may include multiple encoder modules (ECMs) corresponding to each of multiple domains.

[0099] In one embodiment, the encoder module (ECM) may include a first encoder module (ECM) corresponding to a first domain (e.g., boiling point) and a second encoder module (ECM) corresponding to a second domain (e.g., melting point).

[0100] In this embodiment, any one of the multiple encoder modules (ECMs) can be a source encoder module (ECM) that corresponds to the source task of transfer learning according to the embodiment of the present invention.

[0101] Furthermore, of the remaining encoder modules (ECMs) excluding the source encoder module (ECM), any one of them can be a target encoder module (ECM) which corresponds to the target task of transfer learning according to the embodiment of the present invention.

[0102] Furthermore, the Regressor Module (RGM) according to the embodiment of the present invention can be a Head module that takes a predetermined latent vector as input and generates a final predicted value based on the input latent vector.

[0103] Such regression modules (RGMs) can directly participate in the final output generation and determine the predictive performance of the model.

[0104] Furthermore, in this embodiment, the regression module (RGM) may include multiple regression modules (RGMs) corresponding to each of multiple domains.

[0105] As an embodiment, the regressor module (RGM) may include a first regressor module (RGM) corresponding to a first domain (e.g., boiling point) and a second regressor module (RGM) corresponding to a second domain (e.g., melting point).

[0106] In this embodiment, any one of the multiple regression modules (RGMs) can be a source regression module (RGM) that corresponds to the source task of transfer learning according to the embodiment of the present invention.

[0107] Furthermore, of the remaining regression modules (RGMs) excluding the source regression module (RGM), any one of them can be a target regression module (RGM), which is a regression module (RGM) corresponding to the target task of transfer learning according to the embodiment of the present invention.

[0108] Furthermore, a Transfer Module (TFM) according to an embodiment of the present invention can be a module that maps a predetermined latent vector to the latent space of another task and converts it into a Transfer vector.

[0109] Specifically, in an embodiment, the transfer module (TFM) can map a specific latent vector to the latent space of another task based on Riemannian geometry and convert it into a transfer vector.

[0110] In this process, the Transfer Module (TFM) can achieve the geometric alignment between each task being mapped according to the embodiments of the present invention. A detailed explanation of this will be provided later in the multitasking model learning method described below.

[0111] In other words, in an embodiment, the transfer module (TFM) can effectively transfer knowledge data between multiple tasks by mapping the latent vector from the first task to the latent space from the second task via a geometric alignment according to the embodiment of the present invention.

[0112] In this embodiment, the transfer module (TFM) can utilize an autoencoder structure to support data processing that enhances the accuracy and consistency of the converted vector (i.e., the transfer vector).

[0113] Furthermore, in an embodiment, the transfer module (TFM) may include multiple transfer modules (TFMs) corresponding to each of multiple domains.

[0114] As an embodiment, the transfer module (TFM) may include a first transfer module (TFM) corresponding to a first domain (e.g., boiling point) and a second transfer module (TFM) corresponding to a second domain (e.g., melting point).

[0115] In this embodiment, any one of the multiple transfer modules (TFMs) can be a source transfer module (TFM) that corresponds to the source task of transfer learning according to the embodiment of the present invention.

[0116] Furthermore, of the remaining transfer modules (TFMs) excluding the source transfer module (TFM), any one of them can be a target transfer module (TFM), which is a transfer module (TFM) corresponding to the target task of transfer learning according to the embodiment of the present invention.

[0117] Furthermore, an inverse module (ITM: Inverse Transfer Module) according to an embodiment of the present invention may be a module that reconfigures a transfer vector, which has been mapped and transformed into the latent space of another task by a transfer module (TFM), so that it is mapped back into its original latent space.

[0118] As a result, in this embodiment, the inverse module (ITM) can generate a vector (hereinafter referred to as the inverse vector) that has been reconstructed and transformed back to its original state from the transfer vector.

[0119] In this embodiment, the inverse module (ITM) can improve the stability of the reconstruction process described above and the accuracy and consistency of the transfer vector by utilizing an autoencoder structure.

[0120] In an embodiment, such an inverse module (ITM) may include multiple inverse modules (ITMs) corresponding to each of multiple domains.

[0121] As an embodiment, the inverse module (ITM) may include a first inverse module (ITM) corresponding to a first domain (e.g., boiling point) and a second inverse module (ITM) corresponding to a second domain (e.g., melting point).

[0122] In this embodiment, any one of the multiple inverse modules (ITMs) can be a source-inverse module (ITM) that corresponds to the source task of transfer learning according to the embodiment of the present invention.

[0123] Furthermore, of the remaining inverse modules (ITMs) excluding the source inverse module (ITM), any one of them can be a target inverse module (ITM), which is an inverse module (ITM) corresponding to the target task of transfer learning according to the embodiment of the present invention.

[0124] Furthermore, a perturbation module (PBM) according to an embodiment of the present invention can be a module that generates a plurality of perturbation vectors by adding a predetermined change to a predetermined embedding vector.

[0125] Specifically, in the embodiment, the perturbation module (PBM) can be a module that generates a plurality of perturbation vectors (i.e., perturbation points) on the periphery of a particular embedding vector by applying a change that moves the embedding vector in a predetermined direction.

[0126] In this case, the multiple perturbation vectors generated are designed to maintain a relative distance from the corresponding embedding vectors, thereby effectively assisting in geometric alignment.

[0127] In other words, a perturbation module (PBM) like the one described above can help align the coordinate systems between the source and target tasks by generating multiple perturbation vectors to assist in the geometric alignment of the model.

[0128] In addition, in the embodiment, the perturbation module (PBM) can calculate the distance between a predetermined embedding vector and a plurality of perturbation vectors generated therefrom, and assist in matching the displacement between the source task and the target task based on the calculated distance.

[0129] Through this, the Perturbation Module (PBM) can more easily maintain consistency in the latent space for the model.

[0130] Depending on the embodiment, the perturbation module (PBM) can prevent model overfitting and improve generalization performance by forcing a relationship between a given embedding vector and a plurality of perturbation vectors generated therefrom to be maintained.

[0131] Furthermore, the Loss Calculation Module (LCM) according to the embodiment of the present invention can be a module that calculates various loss functions based on various vectors obtained via a multitasking learning model (MtLM).

[0132] In the embodiment, the Loss Calculation Module (LCM) can calculate regression loss, autoencoder loss, consistency loss, mapping loss, distance loss, and / or integrated loss according to the embodiment of the present invention. A detailed explanation of this will be given later in the multitasking model learning method described below.

[0133] Through this, the Loss Calculation Module (LCM) can assist in normalizing and learning different parts of the model, providing feedback for model learning and enabling model optimization.

[0134] On the other hand, in the embodiments of the present invention, the multitasking learning model (MtLM) can perform model optimization and updates through various data processing processes linked with the modules and the like described above.

[0135] For example, a multitasking learning model (MtLM) can perform model optimization and parametric updates in conjunction with the aforementioned modules, based on algorithms such as the AdamW optimization algorithm.

[0136] Thus, in the embodiments of the present invention, the multitasking learning model (MtLM) not only learns knowledge data from various domains simultaneously, but also efficiently learns various inter-domain relationships, thereby expanding the learning domain. At the same time, it enables effective multitasking learning that simultaneously learns local patterns from each domain and common principles across multiple domains.

[0137] As a result, the Multitasking Learning Model (MtLM) can directly improve the processing performance and accuracy of various multitasking tasks based on the model learned as described above.

[0138] [How to implement a multitasking learning model provision service]

[0139] The following describes in detail how a computing system 1000 according to one embodiment of the present invention processes multiple tasks for output from multiple domains by transferring and learning knowledge data from each task's latent space through geometric alignment into a single integrated latent space, and then implementing a multitasking learning model (MtLM) service based on this.

[0140] Generally, existing transfer learning techniques are primarily focused on the classification of image and / or language datasets and have limitations in solving regression problems or problems in non-Euclidean spaces.

[0141] In particular, if the training dataset is insufficient, the decline in predictive performance for the aforementioned problems becomes even more inevitable, and if multitasking that considers various task types is required, the decline in performance in training and prediction for this purpose becomes even more pronounced.

[0142] Furthermore, most existing methods are optimized for handling data in Euclidean space and therefore do not work effectively in complex curvilinear or nonlinear spaces.

[0143] Figure 7 shows an example of a conceptual diagram illustrating a multitasking model learning method according to one embodiment of the present invention.

[0144] Therefore, as shown in Figure 7, the computing system 1000 according to one embodiment of the present invention aims to provide a novel multitasking model learning method that can overcome the regression problem of small datasets and the limitations of existing transfer learning techniques, and a multitasking execution method that utilizes a machine learning model learned therefrom.

[0145] In the following description of one embodiment of the present invention, for the sake of effective explanation, the substance described above will be limited to "molecules," and the domains relating thereto will be described based on "physical properties."

[0146] This takes into account that molecular datasets generally have a small amount of data, include various task types, and primarily deal with regression problems.

[0147] In other words, molecular datasets require various task processing linked to numerous physical properties, but the given data is very restrictive, and each physical property is closely interconnected or influences the others.

[0148] Considering these points, molecular datasets are advantageous for application to multi-task processing involving multiple domains and can serve as a desirable example for explaining a multitasking model learning method according to one embodiment of the present invention and a multitasking execution method using a machine learning model learned therefrom.

[0149] However, it is obvious that the present invention is not limited thereto, and any embodiment that allows for the application of multi-domain multitasking may be included in the embodiments of the present invention.

[0150] The following will describe in more detail, with reference to the attached drawings, a multitasking model learning method according to one embodiment of the present invention and a multitasking execution method using a machine learning model learned therefrom.

[0151] Figure 8 shows a flowchart of blocks illustrating a multitasking model learning method according to one embodiment of the present invention.

[0152] As shown in Figure 8, a multitasking model learning method and a multitasking execution method using a machine learning model learned therefrom, according to one embodiment of the present invention, may include the steps of initializing a multitasking learning model (MtLM) (S101), acquiring experimental data (S103), training the multitasking learning model (MtLM) based on the acquired experimental data (S105), and providing the trained multitasking learning model (MtLM) (S107).

[0153] Specifically, the computing system 1000 according to one embodiment of the present invention can initialize a multitasking learning model (MtLM). (S101)

[0154] In other words, the Geometrically Aligned Transfer Encoder Model (MtLM) according to an embodiment of the present invention can be a machine learning model that aligns fragmented knowledge data (such as latent vectors in an embodiment) in a task-specific latent space M via geometric transfer in order to handle multiple tasks for output from multiple domains.

[0155] In other words, the multitasking learning model (MtLM) according to this embodiment not only learns knowledge data from various domains simultaneously, but also efficiently learns various inter-domain relationships, thereby expanding the learning domain. At the same time, it enables effective multitasking learning that simultaneously learns local patterns from each domain and common principles across multiple domains.

[0156] Specifically, in this embodiment, the computing system 1000 can perform initialization on each component of the multitasking learning model (MtLM) as described above.

[0157] As an embodiment, the computing system 1000 has an embedded network within a multitasking learning model (MtLM) JPEG2026521410000002.jpg1036 ), encoder network ( JPEG2026521410000003.jpg108 ), Regressor (head) network ( JPEG2026521410000004.jpg109 ), Transfer Network ( JPEG2026521410000005.jpg106 ) and / or inverse network ( JPEG2026521410000006.jpg106 ) etc. as random parameter ( JPEG2026521410000007.jpg106 It can be initialized to ).

[0158] Furthermore, as an embodiment, the computing system 1000 can set a predetermined optimization algorithm to be applied to a multitasking learning model (MtLM).

[0159] Exemplary, the computing system 1000 can set the AdamW (Decoupled Weight Decay Regularization) algorithm as the optimization algorithm, and depending on the embodiment, the optimization algorithm can be improved and used to handle weight decay independently.

[0160] Furthermore, the computing system 1000 according to one embodiment of the present invention can acquire experimental data. (S103)

[0161] In other words, the experimental data according to the embodiment of the present invention ( JPEG2026521410000008.jpg108 ) can be data that includes predetermined material-specific property information and material property identification information, used as training data for training a multitasking learning model (MtLM).

[0162] In this embodiment, the material-specific property information can be information that identifies the unique properties possessed by a given substance. That is, in this embodiment, the material-specific property information can be information that identifies the unique properties possessed by a given molecule.

[0163] For example, material-specific property information may include a predetermined material name, molecular structural formula, and / or chemical formula.

[0164] Furthermore, the material property identification information according to the embodiment can be information that identifies the data values ​​that a given material possesses for a given physical property.

[0165] For example, material property identification information may include property (i.e., domain) values ​​such as boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity of a given substance.

[0166] Specifically, in this embodiment, the computing system 1000 can acquire the experimental data described above based on predetermined user input and / or interaction with an external server.

[0167] Furthermore, the computing system 1000 according to one embodiment of the present invention can train a multitasking learning model (MtLM) based on acquired experimental data. (S105)

[0168] Figure 9 shows a block flowchart illustrating a multitasking learning model (MtLM) training method according to one embodiment of the present invention, and Figure 10 shows an example of a conceptual diagram illustrating a multitasking learning model (MtLM) training method according to one embodiment of the present invention.

[0169] In other words, as shown in Figures 9 and 10, in this embodiment, the computing system 1000 can perform pre-training on a multitasking learning model (MtLM) based on the experimental data acquired as described above.

[0170] Specifically, in this embodiment, the computing system 1000 can set up a training loop for a multitasking learning model (MtLM). (S201)

[0171] More specifically, in the embodiment, the computing system 1000 can set the number of epoch iterations, task iterations, and / or batch iterations during training.

[0172] As an embodiment, the computing system 1000 can configure the training loop to repeatedly execute epoch "i" from "1 to n (n>=1)" during training, repeatedly execute for each task "t", and repeatedly execute for each pre-configured arrangement "b".

[0173] Furthermore, in this embodiment, the computing system 1000 can acquire geometrically aligned vectors of the acquired experimental data base, as described above. (S203)

[0174] Here, the geometric alignment vector according to the embodiment of the present invention can mean various vectors obtained through a multitasking learning model (MtLM).

[0175] In one embodiment, the geometric alignment vector is the embedding vector ( JPEG2026521410000009.jpg1010 ), perturbation vector ( JPEG2026521410000010.jpg1015 This may include encoding vectors, transfer vectors, and inverse vectors.

[0176] Specifically, in this embodiment, the computing system 1000 can input the acquired experimental data into a multitasking learning model (MtLM).

[0177] Furthermore, in this embodiment, the computing system 1000 can obtain (1) embedding vectors based on a multitasking learning model (MtLM) to which experimental data has been input.

[0178] More specifically, the computing system 1000 can work in conjunction with the embedding module (EBM) of a multitasking learning model (MtLM) to convert the input experimental data into embedding vectors via an embedding network.

[0179] This allows the computing system 1000 to obtain an embedded vector by projecting the experimental data onto a predetermined embedded space and converting it into a vector format.

[0180] In addition, in this embodiment, the computing system 1000 can generate perturbation vectors based on the acquired embedding vectors.

[0181] Specifically, in this embodiment, the computing system 1000 can work in conjunction with the perturbation module (PBM) of a multitasking learning model (MtLM) to generate a plurality of perturbation vectors (i.e., perturbation points) on a predetermined periphery based on the acquired embedding vector.

[0182] In this embodiment, the computing system 1000 can repeatedly perform the above-described functional operations for each task to obtain a perturbation vector corresponding to each task.

[0183] In one embodiment, the computing system 1000 can obtain a perturbation vector corresponding to task "t" and a perturbation vector corresponding to task "s".

[0184] In addition, in the embodiment, the computing system 1000 can obtain the encoding vectors of the generated perturbation vectors and embedding vectors.

[0185] Here, the encoding vector according to the embodiment may include a perturbation latent vector, which is a latent vector generated based on a predetermined perturbation vector, and an original latent vector, which is an embedding vector, which is the original vector of the perturbation vector.

[0186] Specifically, in this embodiment, the computing system 1000, in conjunction with the encoder module of the multitasking learning model (MtLM), can project the generated perturbation vectors into the latent space corresponding to the task via an encoder network and convert them into latent vectors.

[0187] Furthermore, in this embodiment, the computing system 1000 can work in conjunction with the encoder module of the multitasking learning model (MtLM) to project the acquired embedding vectors into the latent space corresponding to the task via an encoder network, thereby converting them into latent vectors.

[0188] In this embodiment, the computing system 1000 can obtain the perturbation latent vector and the original latent vector.

[0189] In this embodiment, the computing system 1000 repeatedly performs the above-described functional operations for each task and can acquire the original latent vector and perturbation latent vector corresponding to each task.

[0190] As an embodiment, the computing system 1000 has a source latent vector corresponding to task "t" ( JPEG2026521410000011.jpg109 :The following are the original latent vector (t) and the perturbation latent vector corresponding to task "t" ( JPEG2026521410000012.jpg1015 The following can be obtained: (The t-th perturbation latent vector).

[0191] Furthermore, the computing system 1000 generates the original latent vector corresponding to task "s" ( JPEG2026521410000013.jpg109 :The following are the original latent vector (s) and the perturbation latent vector corresponding to task "s" ( JPEG2026521410000014.jpg1014 The following can be obtained (the s-th perturbation latent vector).

[0192] In addition, in this embodiment, the computing system 1000 can acquire the transfer vectors of the acquired encoding vector base.

[0193] Here, the transfer vector according to the embodiment may include a perturbation transfer vector, which is a transfer vector generated based on a predetermined perturbation latent vector, and a source transfer vector, which is a transfer vector generated based on a source latent vector corresponding to the perturbation latent vector.

[0194] Specifically, in an embodiment, the computing system 1000, in conjunction with the transfer module (TFM) of a multitasking learning model (MtLM), can map the acquired perturbation latent vector and original latent vector to the latent space of another task (in an embodiment, task "s" or task "t") via a transfer network and convert them into transfer vectors.

[0195] This allows the computing system 1000 to obtain the perturbation transfer vector and the original transfer vector.

[0196] In this embodiment, the computing system 1000 can repeatedly perform the above-described functional operations for each task and obtain the original transfer vector and perturbation transfer vector corresponding to each task.

[0197] As an embodiment, the computing system 1000 has an original transfer vector corresponding to task "t" ( JPEG2026521410000015.jpg1014 :The following are the original transfer vector (t) and the perturbation transfer vector corresponding to task "t" ( JPEG2026521410000016.jpg1015 The following can be obtained (the t-th perturbation transfer vector).

[0198] Furthermore, the computing system 1000 provides the original transfer vector corresponding to task "s" ( JPEG2026521410000017.jpg1013 :The following are the original transfer vector (s) and the perturbation transfer vector corresponding to task "s" ( JPEG2026521410000018.jpg1016 The following can be obtained: (The sth perturbation transfer vector).

[0199] In this embodiment, the computing system 1000 can acquire geometrically aligned vectors of the experimental data base (i.e., embedding vectors, perturbation vectors, encoding vectors (including original latent vectors and perturbation latent vectors), and transfer vectors (including original transfer vectors and perturbation transfer vectors), etc.).

[0200] Furthermore, in this embodiment, the computing system 1000 can acquire an inverse vector of the acquired transfer vector substrate.

[0201] Here, the inverse vector according to the embodiment may include a perturbation inverse vector, which is an inverse vector generated based on a predetermined perturbation transfer vector, and an original inverse vector, which is an inverse vector generated based on an original transfer vector corresponding to the perturbation transfer vector.

[0202] Specifically, in this embodiment, the computing system 1000, in conjunction with the inverse module (ITM) of the multitasking learning model (MtLM), can reconstruct the acquired perturbation transfer vector and original transfer vector via an inverse network so that they are mapped back to the original latent space and converted into inverse vectors.

[0203] This allows the computing system 1000 to obtain the perturbation inverse vector and the original inverse vector.

[0204] In this embodiment, the computing system 1000 repeatedly performs the above-described functional operations for each task and can obtain the original inverse vector and perturbation inverse vector corresponding to each task.

[0205] As an embodiment, the computing system 1000 provides an original inverse vector corresponding to task "t" ( JPEG2026521410000019.jpg107 :The following are the original inverse vector (t) and the perturbation inverse vector corresponding to task "t" ( JPEG2026521410000020.jpg107 The following can be obtained (the t-th perturbation inverse vector).

[0206] Furthermore, the computing system 1000 generates the original inverse vector corresponding to task "s" ( JPEG2026521410000021.jpg107 :The following are the original inverse vector (s) and the perturbation inverse vector corresponding to task "s" ( JPEG2026521410000022.jpg106 The following can be obtained: (The s-th perturbation inverse vector).

[0207] In this embodiment, the computing system 1000 can acquire geometrically aligned vectors of the experimental data base (i.e., embedding vectors, perturbation vectors, encoding vectors (including original latent vectors and perturbation latent vectors), transfer vectors (including original transfer vectors and perturbation transfer vectors), and inverse vectors (including original inverse vectors and perturbation inverse vectors), etc.).

[0208] Furthermore, in this embodiment, the computing system 1000 can calculate the geometric alignment loss of the acquired geometric alignment vector base. (S205)

[0209] Here, the geometric alignment loss according to the embodiment of the present invention can mean various loss functions calculated based on various vectors (i.e., geometric alignment vectors) obtained via a multitasking learning model (MtLM).

[0210] In one embodiment, the geometric alignment loss is the regression loss ( JPEG2026521410000023.jpg1014 :Regression loss), autoencoder loss ( JPEG2026521410000024.jpg1018 :Auto encoder loss), consistency loss ( JPEG2026521410000025.jpg1018 :Consistency loss), mapping loss ( JPEG2026521410000026.jpg1017 :Mapping loss), distance loss ( JPEG2026521410000027.jpg1013 Distance loss), and / or combined loss ( JPEG2026521410000028.jpg1013 This can include things like integrated loss.

[0211] In the following explanation, for the sake of effective explanation, we will calculate and explain the geometric alignment loss based on task "t".

[0212] Figures 11 and 12 illustrate diagrams illustrating a regression loss calculation method according to one embodiment of the present invention.

[0213] Specifically, as shown in Figures 10 to 12, in this embodiment, the computing system 1000 can calculate 1) regression loss based on a multitasking learning model (MtLM) that has acquired geometrically aligned vectors.

[0214] More specifically, in this embodiment, the computing system 1000 predicts the predicted value via the regression module (RGM) by the following [Equation 1] ( JPEG2026521410000029.jpg107 ) and actual value ( JPEG2026521410000030.jpg108 , in other words, the regression loss can be calculated based on the label value. Here, the predicted value in [Equation 1] is JPEG2026521410000031.jpg1020 It can also be expressed as ".

[0215] [Mathematics 1] JPEG2026521410000032.jpg1688

[0216] In other words, the computing system 1000 can calculate the regression loss by calculating the mean squared error (MSE) between the predicted value and the actual value.

[0217] In this embodiment, each task can prevent mutual interference by calculating an independent regression loss based on the encoder module (ECM) and regression module (RGM) matched to each task, and performing learning based on that loss.

[0218] In this way, the computing system 1000 can easily evaluate the regression performance of the model by calculating the regression loss.

[0219] Furthermore, as shown in Figure 10, in this embodiment, the computing system 1000 can calculate 2) autoencoder loss based on a multitasking learning model (MtLM) that has acquired geometrically aligned vectors.

[0220] Specifically, in this embodiment, the computing system 1000 can calculate the autoencoder loss based on the original latent vector and the original inverse vector using the following [Equation 2].

[0221] [Math 2] JPEG2026521410000033.jpg1692

[0222] In other words, the computing system 1000 can calculate the autoencoder loss by calculating the mean squared error (MSE) between the latent vector and the inverse vector.

[0223] In this embodiment, the computing system 1000 can improve accuracy during the data transfer process through the autoencoder loss calculated as described above.

[0224] Figure 13 shows an example diagram illustrating a mapping method for an integrated latent space M according to one embodiment of the present invention.

[0225] On the other hand, as shown in Figure 13, in this embodiment, the computing system 1000 can learn a bidirectional transformation matrix (TM) that can be mapped to a common integrated latent space M for each task.

[0226] Specifically, in this embodiment, the computing system 1000 can connect the latent spaces between tasks by utilizing knowledge data that holds all the labels for both-sided tasks.

[0227] In this process, the computing system 1000 can calculate the consistency loss and mapping loss according to the embodiment.

[0228] Figures 14 and 15 illustrate diagrams illustrating a consistency loss calculation method according to one embodiment of the present invention.

[0229] More specifically, as shown in Figures 10, 14, and 15, in this embodiment, the computing system 1000 can calculate 3) consistency loss based on a multitasking learning model (MtLM) that has acquired geometrically aligned vectors.

[0230] Specifically, in an embodiment, the computing system 1000 can calculate the consistency loss based on the perturbation transfer vector of task "t" and the perturbation transfer vector of task "s" using the following [Equation 3].

[0231] [Math 3] JPEG2026521410000034.jpg16113

[0232] In other words, the computing system 1000 can calculate the consistency loss by calculating the mean squared error (MSE) between the t-th perturbation transfer vector and the s-th perturbation transfer vector.

[0233] In this embodiment, the computing system 1000 can derive a metric for calculating spatial distance from the transformation matrix (TM), and learn to make the distances identical in the latent space of each task based on the derived metric.

[0234] Through this, the computing system 1000 can more effectively achieve inter-task geometric alignment.

[0235] Figures 16 and 17 illustrate diagrams illustrating a mapping loss calculation method according to one embodiment of the present invention.

[0236] Furthermore, as shown in Figures 10, 16, and 17, in this embodiment, the computing system 1000 can calculate the mapping loss based on a multitasking learning model (MtLM) that has acquired geometrically aligned vectors.

[0237] Specifically, in this embodiment, the computing system 1000 can calculate a mapping loss based on the actual value from task "t" and the value predicted based on the original inverse vector from task "s" using the following [Equation 4].

[0238] [Math 4] JPEG2026521410000035.jpg16117

[0239] In other words, the computing system 1000 can calculate the mapping loss by calculating the mean squared error (MSE) between the actual value of task "t" and the predicted value of task "s" using the original inverse vector.

[0240] In this embodiment, the computing system 1000 can calculate the mapping loss as described above, transfer latent vectors from the latent space of one task to the latent space of the other task, and perform the other task based on the transferred vectors, thereby enabling learning that leads to the latent properties becoming mutually similar.

[0241] Through this, the computing system 1000 can evaluate the predictive performance of vectors transferred to the latent space of other tasks and guide learning in a direction that improves this performance.

[0242] Furthermore, as shown in Figure 10, in this embodiment, the computing system 1000 can calculate the distance loss based on a multitasking learning model (MtLM) that has acquired geometrically aligned vectors.

[0243] Specifically, in an embodiment, the computing system 1000 calculates the distance between the original transfer vector and the perturbation transfer vector for each task using the following [Equations 5 and 6]. JPEG2026521410000036.jpg109 The distance loss between tasks can be calculated based on the transfer vector displacement (hereinafter referred to as "transfer vector displacement").

[0244] More specifically, in an embodiment, the computing system 1000 calculates the distance between the t-th original transfer vector and the t-th permutation transfer vector by task "t" according to the following [Equation 5(a)]. JPEG2026521410000037.jpg106 The following can be used to calculate the t-th transfer vector displacement.

[0245] Furthermore, the computing system 1000 calculates the distance between the sth original transfer vector and the sth permutation transfer vector by task "s" according to the following [Equation 5(b)]. JPEG2026521410000038.jpg107 The following can be used to calculate the displacement of the s-th transfer vector.

[0246] [Number 5] JPEG2026521410000039.jpg3279

[0247] Furthermore, in this embodiment, the computing system 1000 can calculate the distance loss by calculating the mean squared error (MSE) between the t-th transfer vector displacement and the s-th transfer vector displacement using the following [Equation 6].

[0248] [Number 6] JPEG2026521410000040.jpg29114

[0249] Here, "M" in [Equation 6] represents the number of perturbation points.

[0250] In this embodiment, the computing system 1000 can define the t-th transfer vector displacement and the s-th transfer vector displacement as the displacements in the source task and the target task, respectively.

[0251] This allows the computing system 1000 to more easily calculate the distance between the original transfer vector and the perturbation transfer vector by interpreting the displacements of the t-th transfer vector and the s-th transfer vector as being in a flat Euclidean space.

[0252] Therefore, the computing system 1000 can help maintain consistency with respect to the latent space of the model more effectively.

[0253] Figure 18 shows an example diagram illustrating an integrated loss calculation method according to one embodiment of the present invention.

[0254] Furthermore, as shown in Figures 10 and 18, in this embodiment, the computing system 1000 can calculate 6) Integrated loss based on a multitasking learning model (MtLM) that has acquired geometrically aligned vectors.

[0255] Specifically, in this embodiment, the computing system 1000 can calculate an integrated loss by weighting the regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss described above using the following [Equation 7].

[0256] [Number 7] JPEG2026521410000041.jpg13159

[0257] At this time, in the embodiment, the computing system 1000 can apply a weight value for each loss function so as to optimize each loss function for a specific aspect of the model.

[0258] Here, "[Formula 7] JPEG2026521410000042.jpg1010 " is the weight value of the autoencoder loss, " JPEG2026521410000043.jpg106 " is the weight value of the consistency loss, " JPEG2026521410000044.jpg107 " is the weight value of the mapping loss, " JPEG2026521410000045.jpg105 " is the weight value of the distance loss.

[0259] In the embodiment, the computing system 1000 can utilize the weight values as described above, adjust the importance of the loss functions corresponding to each weight value in the model learning process, and update the intermediate variables in the direction of minimizing the integrated loss.

[0260] Returning to FIG. 9, further, in the embodiment, the computing system 1000 can perform model optimization and intermediate variable update based on the geometric alignment loss calculated as described above. (S207)

[0261] Specifically, in the embodiment, the computing system 1000 can perform optimization and intermediate variable update for the multitasking learning model (MtLM) based on the integrated loss described above.

[0262] As an embodiment, the computing system 1000 can calculate a gradient based on the integrated loss for each parameter of a multitasking learning model (MtLM) via backpropagation.

[0263] The computing system 1000 can then perform parametric updates of the multitasking learning model (MtLM) using the calculated gradient and a pre-configured optimization algorithm (e.g., the AdamW (Decoupled Weight Decay Regularization) algorithm).

[0264] This enables the computing system 1000 to achieve geometric alignment loss (particularly integrated loss) based multitasking learning model (MtLM) optimization.

[0265] Thus, in this embodiment, the computing system 1000 can perform multitasking learning model (MtLM) optimization and parametric update learning through a combination of various loss functions calculated from multiple angles.

[0266] In this case, each loss function can easily assist in improving the model's performance by correcting for the accuracy, consistency, and / or distance of the knowledge data mapping.

[0267] Through this, the computing system 1000 can provide improved performance that overcomes the regression problem of small datasets and the limitations of existing transfer learning techniques, while simultaneously realizing a multitasking model that operates more stably and provides improved generalization performance.

[0268] In addition, in this embodiment, the computing system 1000 can terminate the training of the multitasking learning model (MtLM). (S209)

[0269] Specifically, in this embodiment, the computing system 1000 can terminate the multitasking learning model (MtLM) training process described above when it meets a preset training termination condition.

[0270] In one embodiment, the computing system 1000 can terminate the training of the multitasking learning model (MtLM) once it has completed the configured training loop.

[0271] Returning to Figure 8, the computing system 1000 according to one embodiment of the present invention can provide a trained multitasking learning model (MtLM). (S107)

[0272] In other words, in this embodiment, the computing system 1000 can provide the multitasking learning model (MtLM) trained as described above in a predetermined manner.

[0273] As an embodiment, the computing system 1000 can provide a multitasking learning model (MtLM) trained according to an embodiment of the present invention in conjunction with a predetermined application service (e.g., a material synthesis / evaluation service, a material property prediction service, and / or an optimal material recommendation service).

[0274] As a result, the computing system 1000 can effectively support the processing of various multitasking tasks using an improved multitasking learning model (MtLM).

[0275] Thus, in this embodiment, the computing system 1000 can provide an improved performance that overcomes the regression problem of small datasets and the limitations of existing transfer learning techniques, while also providing a more stable multitasking learning model (MtLM) by transferring and learning knowledge data from each task's latent space through a geometric alignment in a single integrated latent space in order to handle multiple tasks for output from multiple domains.

[0276] Through this, the computing system 1000 can demonstrate high generalization performance even in situations where the amount of data given is small, includes various types of tasks, or primarily deals with regression problems, and can provide a transfer learning-based multitasking model that operates stably and robustly.

[0277] In other words, even if there are domains among the multiple domains (physical properties in this embodiment) for which experimental data (training data) is insufficient, the computing system 1000 can provide a multitasking learning model (MtLM) with improved predictive performance based on knowledge distilled through geometric alignment-based transfer learning performed in conjunction with other domains.

[0278] For example, if the computing system 1000 has pre-trained a multitasking learning model (MtLM) based on the first to tenth physical properties for each of a plurality of molecular structural formulas, then upon receiving a first molecular structural formula containing only data for the first to fifth physical properties, it can more accurately predict the data values ​​for the remaining sixth to tenth physical properties for the first molecular structural formula based on the knowledge data transferred and distilled through pre-training, and generate and provide output data based on these predictions.

[0279] In this way, the computing system 1000 according to the embodiment of the present invention can provide a multi-tasking model that realizes effective transfer learning of a geometric alignment basis, guarantees high generalization performance, improves the prediction accuracy for regression problems, supports normalization by combining various loss functions, performs a stable learning process, and guarantees robust performance.

[0280] As described above, the multi-tasking model learning method according to an embodiment of the present invention and the multi-tasking execution method using the machine learning model learned based on this can transfer the knowledge learned in the source task to the target task through transfer learning to solve the problem of insufficient data, thereby providing a multi-tasking model that maintains high performance even with a small-scale dataset.

[0281] Therefore, the multi-tasking model learning method according to an embodiment of the present invention and the multi-tasking execution method using the machine learning model learned based on this can expand its application range to fields where data and domain knowledge are insufficient and it is difficult to apply a machine learning model.

[0282] In addition, the multi-tasking model learning method according to an embodiment of the present invention and the multi-tasking execution method using the machine learning model learned based on this can provide a specialized transfer learning technique that can be effectively applied to regression problems, thereby achieving high prediction performance even in complex regression problems such as molecular datasets.

[0283] [[ID=X]] In addition, the multi-tasking model learning method according to an embodiment of the present invention and the multi-tasking execution method using the machine learning model learned based on this can optimize the knowledge transfer between the source task and the target task through Riemannian geometric proximity, thereby maintaining the geometric consistency between tasks and improving the efficiency of transfer learning.

[0284] Furthermore, the multitasking model learning method according to one embodiment of the present invention and the multitasking execution method using the machine learning model learned therefrom have the effect of further improving the generalization performance of the model by combining multiple loss functions and normalizing various aspects of the model.

[0285] Therefore, the multitasking model learning method according to one embodiment of the present invention and the multitasking execution method using the machine learning model learned therefrom have the effect of providing a multitasking model that can be used universally for various substances (materials), thereby improving the quality of related industries as a whole.

[0286] On the other hand, embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., individually or in combination. The program instructions recorded on the computer-readable recording medium may be specifically designed and configured for the present invention or may be publicly known and usable by those skilled in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code, such as that produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. Hardware devices can be modified into one or more software modules to perform the processing according to the present invention, and vice versa.

[0287] The specific embodiments described in this invention are merely examples and do not limit the scope of the invention in any way. For the sake of brevity, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Furthermore, connections such as lines or connecting members between components shown in the drawings are illustrative examples of functional and / or physical or circuit connections and may be substituted or shown as various additional functional, physical, or circuit connections in actual devices. In addition, components that are not necessarily required for the application of the invention may not be required unless specifically mentioned, such as "essential" or "important."

[0288] Furthermore, while the detailed description of the present invention has been provided with reference to preferred embodiments, a person skilled in the art or with commercial knowledge of the art will understand that the present invention can be modified and altered in various ways without departing from the spirit and technical domain of the invention as described in the claims below. Therefore, the technical scope of the present invention should not be limited to what is described in the detailed description of the specification, but should be determined by the claims. Industrial Applicability

[0289] This invention relates to a multitasking model learning method and a multitasking execution method using a machine learning model learned therefrom, and has industrial applicability as it can be used in the artificial intelligence industry.

Claims

1. A method for training a multitasking model in a computing system comprising memory and a processor, The steps include initializing a multitasking model that handles multiple tasks based on multiple domains, and Steps include obtaining predetermined experimental data, The steps include training the multitasking model based on the experimental data obtained, Includes, The steps for training the aforementioned multitasking model are: The steps include obtaining a geometric alignment vector, which is a vector that supports geometric alignment between data in a single unified latent space (Manifold) based on the aforementioned experimental data, and The steps include: calculating a geometric alignment loss based on the acquired geometric alignment vector; The steps include updating the parameters of the multitasking model based on the calculated geometric alignment loss, A multitasking model learning method that includes this.

2. The step of obtaining the aforementioned geometric alignment vector is: A multitasking model learning method according to claim 1, comprising the step of obtaining an embedding vector obtained by projecting the experimental data onto a predetermined embedding space and converting it into a vector format, based on an embedding module included in the multitasking model.

3. The step of obtaining the aforementioned geometric alignment vector is: A method for learning a multitasking model according to claim 2, further comprising the step of obtaining a plurality of perturbation vectors obtained by moving the embedding vector in a predetermined direction based on a perturbation module included in the multitasking model.

4. The step of obtaining the aforementioned geometric alignment vector is: The steps include: obtaining an original latent vector by projecting the embedding vector onto the latent space of the first task (Task 1) based on the encoder module included in the multitasking model; The steps include obtaining a perturbation latent vector by projecting the perturbation vector onto the latent space of the first task, based on the encoder module included in the multitasking model, The multitasking model learning method according to claim 3, further comprising:

5. The step of obtaining the aforementioned geometric alignment vector is: The steps include obtaining an original transfer vector by mapping the original latent vector to the latent space of the second task (Task 2) based on the transfer module included in the multitasking model, The steps include obtaining a perturbation transfer vector by mapping the perturbation latent vector to the latent space of a second task based on the transfer module included in the multitasking model, The multitasking model learning method according to claim 4, further comprising:

6. The step of obtaining the aforementioned geometric alignment vector is: The steps include obtaining an Original inverse vector by remapping the Original transfer vector to the latent space of the first task based on the Inverse Transfer Module included in the multitasking model, The steps include obtaining a perturbation inverse vector by remapping the perturbation transfer vector to the latent space of the first task based on the inverse module included in the multitasking model, The multitasking model learning method according to claim 5, further comprising:

7. The step of calculating the geometric alignment loss is: The steps include calculating regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss based on the geometric alignment vector, The steps include calculating an integrated loss by weighting the regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss calculated above, A multitasking model learning method according to claim 6, including the following:

8. The step of updating the parameters of the multitasking model is: A multitasking model learning method according to claim 7, comprising the step of updating the parameter in a direction that minimizes the combined loss.

9. The step of calculating the geometric alignment loss is: The steps include obtaining predicted values ​​using the latent vector based on the Regressor Module included in the multitasking model, The steps include: calculating the mean squared error based on the obtained predicted value and the label value corresponding to the latent vector; The steps include: calculating the regression loss based on the mean squared error calculated above; The multitasking model learning method according to claim 7, further comprising:

10. The step of calculating the geometric alignment loss is: A step of calculating the mean squared error based on the original latent vector and the original inverse vector, The steps include: calculating the autoencoder loss based on the mean squared error calculated above; The multitasking model learning method according to claim 7, further comprising:

11. The step of calculating the geometric alignment loss is: A step of calculating the mean squared error based on a perturbation transfer vector mapped from the latent space of the first task to the latent space of the second task and a perturbation transfer vector mapped from the latent space of the second task to the latent space of the first task, A step of calculating the consistency loss based on the mean squared error calculated above, The multitasking model learning method according to claim 7, further comprising:

12. The step of calculating the geometric alignment loss is: A step of calculating the mean squared error based on the label value based on the first task and the predicted value based on the original inverse vector based on the second task, A step of calculating the mapping loss based on the mean squared error calculated above, The multitasking model learning method according to claim 7, further comprising:

13. The step of calculating the geometric alignment loss is: A step of calculating a first transfer vector displacement, which is the distance between the original transfer vector and the perturbation transfer vector based on the first task described above, A step of calculating a second transfer vector displacement, which is the distance between the original transfer vector and the perturbation transfer vector based on the second task described above, A step of calculating the mean squared error based on the calculated first transfer vector displacement and the second transfer vector displacement, A step of calculating the distance loss based on the mean squared error calculated above, The multitasking model learning method according to claim 7, further comprising:

14. The aforementioned experimental data A multitasking model learning method according to claim 1, comprising material-specific property information, which is information that identifies the unique properties possessed by a given substance, and material property identification information, which is information that identifies the data values ​​that a given substance possesses for a given physical property.

15. At least one memory, A processor that reads at least one application stored in the memory and trains a multitasking model, Equipped with, The instruction words of the aforementioned processor are: The steps include initializing a multitasking model that handles multiple tasks based on multiple domains, and Steps include obtaining predetermined experimental data, The steps include obtaining a geometric alignment vector, which is a vector that supports geometric alignment between data in a single unified latent space (Manifold) based on the experimental data obtained, and The steps include: calculating a geometric alignment loss based on the acquired geometric alignment vector; The steps include updating the parameters of the multitasking model based on the calculated geometric alignment loss, A multitasking model learning server that includes commands to perform the following actions.

Images