Method for training multi-tasking model according to data-driven technique and system therefor

Abstract

According to an embodiment of the present invention, a method for training a multi-tasking model according to a data-driven technique is a method for training a multi-tasking model according to a data-driven technique by a computing system including a memory and a processor, the method comprising the steps of: performing first optimization for updating parameters of a multi-tasking model on the basis of predetermined experimental data; acquiring validation data that is data for validating predictive performance of the first optimized multi-tasking model; performing second optimization for updating a learning rate of the first optimized multi-tasking model on the basis of the acquired validation data; and providing the second optimized multi-tasking model.

Description

A method and system for learning a multi-task model based on data-centric techniques.

[0001] The present invention relates to a multi-task model learning method and system using data-driven techniques. More specifically, the present invention relates to a multi-task model learning method and system that automatically determines learning rates among multiple tasks using data-driven techniques.

[0002] Machine learning and artificial intelligence models require massive amounts of data. However, realistically, there are always limitations to acquiring sufficient data. This becomes even more acute when applying models to new domains or tasks. Molecular structure datasets are a prime example of this situation. In the fields of chemistry and pharmacology, data is needed to predict the properties of new molecules, but experimental data for each molecule is difficult and expensive to obtain. Therefore, the need for transfer learning techniques, which apply knowledge from previously trained models to new tasks, has increased.

[0003] However, existing transfer learning has been developed primarily focusing on classification problems of large-scale data sets such as image or text data. Therefore, existing transfer learning techniques show limitations when applied to small-scale, complex data sets such as regression problems or molecular data sets. In particular, because molecular structure data is high-dimensional and the relationship between each composition and bond significantly affects each material property, when transfer learning techniques are applied to the relationship between one material property and molecular structure data and then to another material property and molecular structure data, existing Euclidean space-based transfer learning techniques cannot effectively handle such complex structures in non-Euclidean spaces.

[0004] Meanwhile, Riemannian geometry enables calculus in curved spaces, allowing for better representation and analysis of the complex structure of data. This Riemannian approach assumes that latent vectors exist on a curved manifold, which is advantageous for aligning the geometry between complex source and target tasks.

[0005] On the other hand, existing transfer learning methods require manual setting of learning coefficients (i.e., transfer ratios) between source and target tasks, which not only reduces the accuracy of the set learning coefficients but also leads to inefficient and suboptimal results for large-scale task processing.

[0006] Therefore, based on the above background, it is necessary to introduce a new technique that implements efficient optimization of learning coefficients, demonstrates high prediction performance and stability even on small data sets, implements more effective transfer learning, and improves model regularization performance to enhance generalization performance.

[0007] One embodiment of the present invention provides a multi-tasking model learning method and system that automatically determines learning rates between multiple tasks using a data-driven technique.

[0008] At this time, one embodiment of the present invention provides a multi-task model learning method and system that automatically performs a predetermined data-centric learning coefficient update according to a two-level optimization (Bi-level optimization) including a first optimization performed in a pre-learning process and a second optimization performed in a model verification process.

[0009] In addition, one embodiment of the present invention provides a multi-task model learning method and system according to a data-centric technique that performs transfer learning through geometric alignment in an integrated latent space for multiple tasks according to multiple domains during the pre-learning.

[0010] In addition, one embodiment of the present invention provides a multi-task model learning method and system based on a data-centric technique that simultaneously trains various prediction tasks according to multiple domains in the transfer learning process, thereby learning not only individual principles of each domain but also correlations between domains and common principles for the entire domain.

[0011] In addition, one embodiment of the present invention provides a multi-tasking model learning method and system according to a data-centric technique that implements exchange of mutual information by matching geometric characteristics between the various prediction tasks.

[0012] In addition, one embodiment of the present invention provides a multi-task model learning method and system according to a data-centric technique that secures a data set for training by utilizing source data from various sources.

[0013] In addition, one embodiment of the present invention applies the multi-tasking learning model learned as above to predict relationships between multiple properties and materials, thereby providing a multi-tasking model capable of predicting multiple properties for a specific material and predicting a specific material satisfying multiple properties.

[0014] However, the technical problems to be solved by the present invention and embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

[0015] A method for learning a multi-tasking model using a data-centric technique according to an embodiment of the present invention is a method for learning a multi-tasking model using a data-centric technique in a computing system including a memory and a processor, the method comprising: performing a first optimization for updating parameters of a multi-tasking model based on predetermined experimental data; obtaining validation data, which is data for validating the prediction performance of the first optimized multi-tasking model; performing a second optimization for updating a learning rate of the first optimized multi-tasking model based on the obtained validation data; and providing the second optimized multi-tasking model.

[0016] In another aspect, the step of performing the first optimization includes the steps of obtaining a geometric alignment vector, which is a vector that supports geometric alignment in the integrated latent space based on the experimental data, calculating a geometric alignment loss based on the obtained geometric alignment vector, and updating the parameters of the multi-task model based on the calculated geometric alignment loss.

[0017] In another aspect, the learning coefficient is a hyperparameter that determines the transfer ratio between a pair of a source task, which is a task that transfers data, and a target task, which is a task that receives data, when performing transfer learning in the first optimization process.

[0018] In another aspect, the step of performing the second optimization includes the step of calculating a validation loss, which is an indicator for evaluating the generalization performance of the multi-task model, based on the parameters updated based on the first optimization and the validation data, and the step of updating the learning coefficients based on the calculated validation loss.

[0019] In another aspect, the step of updating the learning coefficients includes a step of adjusting the learning coefficients in a direction that minimizes the verification loss.

[0020] In another aspect, the method for learning a multi-task model using a data-centric technique according to an embodiment of the present invention further includes a step of repeatedly performing the first optimization and the second optimization based on a predetermined loop algorithm.

[0021] In another aspect, the experimental data and the verification data include material-specific characteristic information, which is information specifying the unique characteristics of a given material, and material property-specific information, which is information specifying characteristic values ​​that the material has for a plurality of physical properties.

[0022] In another aspect, a method for learning a multi-tasking model using a data-centric technique according to an embodiment of the present invention further includes a step of acquiring property relationship information, which is information including data specifying properties associated with a given property, data specifying an attribute of a relationship between the associated properties, and data specifying a degree of association according to an attribute of the relationship.

[0023] In another aspect, the multi-tasking model learning method according to the data-centric technique according to an embodiment of the present invention further includes a step of updating the learning coefficient based on the property relationship information.

[0024] Meanwhile, a system for learning a multi-tasking model using a data-centric technique according to an embodiment of the present invention comprises: at least one memory; and at least one processor for reading out at least one application stored in the memory and learning a multi-tasking model using a data-centric technique; wherein instructions of the processor include instructions for performing a first optimization step for updating parameters of a multi-tasking model based on predetermined experimental data, a step for obtaining validation data which is data for validating the prediction performance of the first optimized multi-tasking model, a step for performing a second optimization step for updating a learning rate of the first optimized multi-tasking model based on the obtained validation data, and a step for providing the second optimized multi-tasking model.

[0025] A multi-tasking model learning method and system using a data-driven technique according to one embodiment of the present invention automatically determines learning rates between multiple tasks using a data-driven technique, thereby effectively replacing a hyperparameter setting method that relies on manual design, thereby having the effect of deriving optimal learning coefficients (i.e., transfer rates) more accurately and efficiently.

[0026] Therefore, the multi-task model learning method and system using a data-centric technique according to one embodiment of the present invention have the effect of implementing training convergence acceleration and performance improvement in transfer learning for multi-task learning.

[0027] In addition, the method and system for learning a multi-task model using a data-centric technique according to one embodiment of the present invention automatically performs a predetermined data-centric learning coefficient update according to a two-level optimization (Bi-level optimization) including a first optimization performed in a pre-learning process and a second optimization performed in a model verification process, thereby supporting automated updates for learning coefficients that increase exponentially in a large-scale task space, thereby improving model scalability.

[0028] In addition, the method and system for learning a multi-tasking model using a data-centric technique according to one embodiment of the present invention have the effect of minimizing the gap between the learning coefficient predicted based on the tasks and the actual data distribution when the given source task and target task pair do not represent the data population corresponding to the tasks.

[0029] In addition, the method and system for learning a multi-task model using a data-centric technique according to one embodiment of the present invention have the effect of accurately predicting an integrated output that satisfies each requirement of a plurality of domains by performing transfer learning through geometric alignment in an integrated latent space for multi-tasks according to a plurality of domains during the pre-learning.

[0030] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention can simultaneously train various prediction tasks according to multiple domains in the transfer learning process, thereby learning not only individual principles of each domain but also correlations between domains and common principles for the entire domain, thereby expanding the model learning area and simultaneously expanding the prediction acceptance range for each domain and domain.

[0031] Accordingly, the multitasking model learning method and system using a data-centric technique according to one embodiment of the present invention have the effect of directly improving the performance and quality of processing various multitasking tasks using the learned model.

[0032] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of easily supporting the transfer of knowledge between interrelated data and the improvement of prediction performance accordingly by implementing the exchange of mutual information by matching geometric characteristics between the various prediction tasks.

[0033] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of increasing resistance to unnecessary interference information and increasing the stability of the model.

[0034] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention can increase the diversity of data used for model learning and improve learning performance by enabling the model to learn more information by securing a data set for training using source data from various sources.

[0035] In addition, the method for learning a multi-tasking model using a data-centric technique according to an embodiment of the present invention and the system thereof can predict multiple properties for a specific substance by applying the learned multi-tasking learning model as described above to predict relationships between multiple properties and materials, and provide a multi-tasking model capable of predicting a specific substance satisfying multiple properties, thereby providing a multi-tasking model that can be universally utilized for various substances (materials), thereby having the effect of improving the quality of the entire related industry.

[0036] A method and system for learning a multi-tasking model using a data-centric technique according to one embodiment of the present invention can provide a multi-tasking model that maintains high performance for multiple tasks even on a small data set by transferring knowledge learned in a source task to a target task through transfer learning to resolve the problem of insufficient data.

[0037] Therefore, the multi-tasking model learning method and system using a data-centric technique according to one embodiment of the present invention have the effect of expanding the scope of application to fields where it was difficult to apply machine learning models due to insufficient data or domain knowledge.

[0038] In addition, the multi-task model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of demonstrating high prediction performance even in complex regression problems such as molecular data sets by providing a specialized transfer learning technique that can be effectively applied to regression problems.

[0039] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of improving the efficiency of transfer learning by maintaining geometric consistency between tasks by optimizing knowledge transfer between source tasks and target tasks through a Riemannian geometric approach.

[0040] In addition, the multi-task model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of further improving the generalization performance of the model by combining multiple loss functions to normalize various aspects of the model.

[0041] However, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood from the description below.

[0042] FIG. 1 illustrates an example block diagram of a computing system implementing a multitasking model learning service according to one embodiment of the present invention.

[0043] FIG. 2 illustrates an example block diagram of a computing device implementing a multitasking model learning service according to one embodiment of the present invention.

[0044] FIG. 3 illustrates an example block diagram of another aspect of a computing device implementing a multitasking model learning service according to one embodiment of the present invention.

[0045] Figures 4 and 5 illustrate examples of conceptual diagrams for explaining a multitasking learning model according to one embodiment of the present invention.

[0046] FIG. 6 illustrates an example of a conceptual diagram of a multi-task learning model for predicting multiple property values ​​for a specific material according to one embodiment of the present invention.

[0047] Figure 7 illustrates an internal block diagram of a multitasking learning model according to one embodiment of the present invention.

[0048] FIG. 8 illustrates an example of a conceptual diagram for explaining a multitasking learning model including a plurality of task processing units according to one embodiment of the present invention.

[0049] FIG. 9 illustrates an example of a conceptual diagram for explaining a multi-tasking model pre-learning method according to one embodiment of the present invention.

[0050] FIG. 10 illustrates a block flow diagram for explaining a multi-tasking model pre-training method according to one embodiment of the present invention.

[0051] Figure 11 illustrates an example of a knowledge graph representing relationships between properties according to one embodiment of the present invention.

[0052] FIG. 12 illustrates a block flow diagram for explaining a multi-tasking learning model training method according to one embodiment of the present invention.

[0053] FIG. 13 illustrates an example of a first conceptual diagram for explaining a multi-tasking learning model training method according to one embodiment of the present invention.

[0054] FIG. 14 illustrates an example of a second conceptual diagram for explaining a multi-tasking learning model training method according to one embodiment of the present invention.

[0055] FIG. 15 and FIG. 16 illustrate examples of drawings for explaining a method for calculating regression loss according to one embodiment of the present invention.

[0056] FIG. 17 illustrates an example of a drawing for explaining an integrated latent space mapping method according to one embodiment of the present invention.

[0057] FIG. 18 and FIG. 19 illustrate examples of drawings for explaining a consistency loss calculation method according to one embodiment of the present invention.

[0058] FIG. 20 and FIG. 21 illustrate examples of drawings for explaining a mapping loss calculation method according to one embodiment of the present invention.

[0059] FIG. 22 illustrates an example of a drawing for explaining an integrated loss calculation method according to one embodiment of the present invention.

[0060] FIG. 23 illustrates a flowchart for explaining a multi-tasking model learning method based on a data-centric technique based on property relationship information according to one embodiment of the present invention.

[0061] Figure 24 illustrates a conceptual diagram for explaining a multi-tasking model learning method based on a data-centric technique based on property relationship information according to one embodiment of the present invention.

[0062] FIG. 25 illustrates a flowchart for explaining a multi-tasking model learning method based on a data-centric technique based on verification data according to one embodiment of the present invention.

[0063] FIG. 26 illustrates a first conceptual diagram for explaining a multi-tasking model learning method based on a data-centric technique based on verification data according to one embodiment of the present invention.

[0064] FIG. 27 illustrates a second conceptual diagram for explaining a multi-tasking model learning method based on a data-centric technique based on verification data according to one embodiment of the present invention.

[0065] The present invention is capable of various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, as well as the methods for achieving them, will become clear with reference to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms. In the following embodiments, terms such as first, second, etc. are not used in a limiting sense but are used for the purpose of distinguishing one component from another. Furthermore, the singular expression includes plural expressions unless the context clearly indicates otherwise. Furthermore, terms such as "include" or "have" indicate the presence of a feature or component described in the specification, and do not preemptively exclude the possibility that one or more other features or components may be added. Furthermore, in the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to what is shown.

[0066] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals and redundant descriptions thereof will be omitted.

[0067]

[0068] [An exemplary system implementing a multi-tasking model learning service]

[0069] Hereinafter, an exemplary system providing a multi-task model learning service that automatically determines learning coefficients between multiple tasks using a data-driven technique is described in detail with reference to the attached drawings.

[0070] FIG. 1 illustrates an example block diagram of a computing system implementing a multitasking model learning service according to one embodiment of the present invention.

[0071] Referring to FIG. 1, a computing system (1000) implementing a multi-tasking model learning service of the present invention includes a user computing device (110), a server computing system (130), and a training computing system (150), and the devices are capable of communicating via a network (170).

[0072] The method and system for learning a multi-tasking model using a data-centric technique according to one embodiment of the present invention may 1) be implemented and provided locally by a user computing device (110), 2) be implemented and provided in the form of a web service by a server computing system (130) communicating with the user computing device (110), or 3) be implemented and provided by the user computing device (110) and the server computing system (130) in connection with each other.

[0073] At this time, in the embodiment, the user computing device (110) and / or the server computing system (130) can train the machine learning model (120 and / or 140) through interaction with a training computing system (150) communicatively connected via a network (170). The training computing system (150) may be separate from the server computing system (130) or may be part of the server computing system (130).

[0074] And at this time, the artificial intelligence model can be 1) trained directly locally by the user computing device (110), 2) trained by the server computing system (130) and the user computing device (110) interacting with each other through a network (170), and 3) trained by a separate training computing system (150) using various training techniques and learning techniques. And the artificial intelligence model trained by the training computing system (150) can be implemented in a manner of being provided / updated by transmitting the model to the user computing device (110) and / or the server computing system (130) through a network (170).

[0075] In some embodiments, the training computing system (150) may be part of a server computing system (130) or part of a user computing device (110).

[0076] The user computing device (110) may include any type of computing device, such as a smart phone, a mobile phone, a digital broadcasting device, a personal digital assistant (PDA), a portable multimedia player (PMP), a desktop, a wearable device, an embedded computing device, and / or a tablet PC.

[0077] Such a user computing device (110) includes at least one processor (111) and memory (112). Here, the processor (111) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0078] The memory (112) may include one or more non-transitory / transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof, and may include web storage of a server that performs a memory storage function on the Internet. The memory (112) may store data (113) and commands (114) necessary for the at least one processor (111) to perform functional operations, such as training an artificial intelligence model or executing multi-tasking learning through an artificial intelligence model.

[0079] In one embodiment, the user computing device (110) may store at least one machine learning model (120).

[0080] In detail, the machine learning model (120) may be various machine learning models such as multiple neural networks (e.g., deep neural networks) or other types of machine learning models including nonlinear models and / or linear models, and may be composed of a combination thereof.

[0081] At this time, the neural network may include at least one of feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, and / or other types of neural networks.

[0082] In one embodiment, the user computing device (110) may receive at least one machine learning model (120) from a server computing system (130) via a network (170), store the model in a memory (112), and then execute the stored machine learning model (120) by a processor (111) to perform multi-tasking learning, etc.

[0083] In another embodiment, the server computing system (130) may include at least one machine learning model (140), perform operations through the machine learning model (140), and communicate data related thereto with the user computing device (110) in conjunction with the user computing device (110) to provide a multi-tasking model learning service to the user.

[0084] For example, a user computing device (110) can perform a multi-tasking model learning service in a manner that a server computing system (130) provides output for a user's input using a machine learning model (140) via the web.

[0085] Additionally, the artificial intelligence model may be implemented in such a way that at least some of the machine learning models (120 and / or 140) are executed on a user computing device (110) and the rest are executed on a server computing system (130).

[0086] In addition, the user computing device (110) may include at least one input component (121) that detects a user's input. For example, the user input component (121) may include a touch sensor (e.g., a touch screen and / or a touch pad, etc.) that detects a touch of a user's input medium (e.g., a finger or a stylus), an image sensor that detects a user's motion input, a microphone, a button, a mouse, and / or a keyboard that detects a user's voice input, etc. In addition, the user input component (121) may include an interface and an external controller when receiving an input to an external controller (e.g., a mouse and / or a keyboard, etc.) through an interface.

[0087] The server computing system (130) includes at least one processor (131) and memory (132). Here, the processor (131) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0088] And the memory (132) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. This memory (132) may store data (133) and instructions (134) necessary for the processor (131) to perform functional operations such as training an artificial intelligence model or executing multi-tasking learning through an artificial intelligence model.

[0089] In one embodiment, the server computing system (130) may be implemented to include at least one computing device. For example, the server computing system (130) may be implemented to operate multiple computing devices according to a sequential computing architecture, a parallel computing architecture, or a combination thereof. Additionally, the server computing system (130) may include multiple computing devices connected via a network (170).

[0090] Additionally, the server computing system (130) may store at least one machine learning model (140). For example, the server computing system (130) may include a neural network and / or other multi-layer nonlinear model as the machine learning model (140). Exemplary neural networks may include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0091] The training computing system (150) includes at least one processor (151) and memory (152). Here, the processor (151) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0092] And the memory (152) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. This memory (152) may store data (153) and instructions (154) necessary for the processor (151) to perform learning of an artificial intelligence model, etc.

[0093] For example, the training computing system (150) may include a model trainer (160) that trains a machine learning model (120 and / or 140) stored in the user computing device (110) and / or server computing system (130) using various training or learning techniques, such as backpropagation of errors (according to the framework illustrated in FIG. 3).

[0094] For example, such a model trainer (160) may perform updates to one or more parameters of a machine learning model (120 and / or 140) in a backpropagation manner based on a defined loss function.

[0095] In some implementations, performing backward propagation of errors may include performing truncated backpropagation through time. The model trainer (160) may perform a number of generalization techniques (e.g., weight reduction, dropout, and / or knowledge distillation) to improve the generalization ability of the trained machine learning model (120 and / or 140).

[0096] In particular, the model trainer (160) can train the machine learning model (120 and / or 140) based on a series of training data (161). Here, the training data (161) can include data in different formats, such as, for example, images, audio samples, and / or text. Examples of image types that can be used can include video frames, LiDAR point clouds, X-ray images, computed tomography scans, hyperspectral images, and / or various other forms of images.

[0097] Such training data (161) may be provided by a user computing device (110) and / or a server computing system (130). When the training computing device trains a machine learning model (120 and / or 140) on specific data of the user computing device (110), the machine learning model (120 and / or 140) may be characterized as a personalized model.

[0098] And the model trainer (160) includes computer logic utilized to provide the desired function.

[0099] Additionally, the model trainer (160) may be implemented as hardware, firmware, and / or software that controls a general-purpose processor. In one implementation, the model trainer (160) includes a program file stored in a storage device, which may be loaded into a memory (152) and executed by one or more processors (151). In another implementation, the model trainer (160) includes one or more sets of computer-executable data (153) and instructions (154) stored in a tangible computer-readable storage medium, such as a RAM hard disk or an optical or magnetic medium.

[0100] The network (170) includes, but is not limited to, a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), a Wireless Local Area Network (Wireless LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, and / or a Digital Multimedia Broadcasting (DMB) network.

[0101] In general, communication over a network (170) may be performed using any type of wired and / or wireless connection, using various communication protocols (e.g., TCP / IP, HTTP, SMTP, and / or FTP), encodings or formats (e.g., HTML and / or XML), and / or protection schemes (e.g., VPN, Secure HTTP, and / or SSL).

[0102] FIG. 2 illustrates an example block diagram of a computing device implementing a multitasking model learning service according to one embodiment of the present invention.

[0103] As shown in FIG. 2, the computing device (100) included in the user computing device (110), the server computing system (130), and the training computing system (150) includes a plurality of applications (e.g., Application 1 to Application N). Each application may include a machine learning library and one or more machine learning models. For example, the applications may include an image processing (e.g., Detection, Classification, and / or Segmentation, etc.) application, a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and / or a chat-bot application.

[0104] In an embodiment, the computing device (100) may include a model trainer (160) for training an artificial intelligence model, and may store and operate the trained artificial intelligence model to provide output data according to predetermined input data (for example, material characteristic information and / or material property information, etc.).

[0105] Each application of the computing device (100) may communicate with a number of other components of the computing device (100), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In one embodiment, each application may communicate with each device component using an API (e.g., a public API). In one embodiment, the API used by each application may be specific to that application.

[0106] FIG. 3 illustrates an example block diagram of another aspect of a computing device implementing a multitasking model learning service according to one embodiment of the present invention.

[0107] Referring to FIG. 3, a computing device (200) includes a plurality of applications (e.g., Application 1 to Application N). Each application may communicate with a central intelligence layer. For example, the applications may include an image processing application, a text messaging application, an email application, a dictation application, a virtual keyboard application, and / or a browser application. In one embodiment, each application may communicate with the central intelligence layer (and models stored therein) using an API (e.g., a common API across all applications).

[0108] The central intelligence layer may include multiple machine learning models. For example, as illustrated in FIG. 3, at least some of the machine learning models may be provided to each application and managed by the central intelligence layer. In other implementations, two or more applications may share a single machine learning model. For example, in some implementations, the central intelligence layer may provide a single model to all applications. In some implementations, the central intelligence layer may be included within the operating system of the computing device (200) or implemented differently.

[0109] The central intelligence layer may communicate with a central device data layer. The central device data layer may be a centralized data repository for the computing device (200). As illustrated in FIG. 3, the central device data layer may communicate with a number of other components of the computing device (200), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer may communicate with each device component using an API (e.g., a private API).

[0110] The techniques described herein may refer to servers, databases, software applications, and other computer-based systems, as well as actions taken and information transmitted to or from such systems. It will be appreciated that the inherent flexibility of computer-based systems allows for a wide range of possible configurations, combinations, and division of labor and functionality between and among components. For example, the processes described herein may be implemented using a single device or component, or multiple devices or components operating in combination. Databases and applications may be implemented on a single system or in a distributed system across multiple systems. Distributed components may operate sequentially or in parallel.

[0111]

[0112] Multitasking Learning Model (MtLM)

[0113] FIGS. 4 and 5 illustrate examples of conceptual diagrams for explaining a multi-task learning model (MtLM) according to one embodiment of the present invention.

[0114] Referring to FIGS. 4 and 5, a multi-task learning model (MtLM) (Geometrically Aligned Transfer Encoder Model) according to an embodiment of the present invention may be a machine learning model that aligns fragmented knowledge data (for example, latent vectors, etc.) in each task's latent space through geometric transfer in a single integrated latent space (M: Manifold) in order to process multi-tasks for an integrated output satisfying multiple domains.

[0115] Here, for reference, the latent space according to the embodiment may refer to a high-dimensional space where certain data is located after being transformed through an encoding process. In this space, important data characteristics can be expressed in a compressed form.

[0116] Meanwhile, the integrated latent space (M) according to the embodiment is a virtual local plane (LF) space that geometrically expresses a given data, and the transformation between the single latent space and the integrated latent space (M) may serve to match the geometric characteristics of the data.

[0117] Returning to the above, the multi-task learning model (MtLM) according to the embodiment can perform effective multi-task learning that expands the learning area by simultaneously learning knowledge data according to various domains and efficiently learning relationships between multiple domains, while simultaneously implementing batch learning of local patterns according to each domain and common principles across multiple domains.

[0118] Accordingly, the multi-tasking learning model (MtLM) can directly improve the processing performance and accuracy of various multi-tasking tasks based on the model learned as above.

[0119] In the following examples, a multi-tasking learning model (MtLM) will be described as an example of a relationship between a given material and multiple properties, and will be described as a learning model that multi-tasks multiple tasks, including a first task of predicting the characteristics of a first material property for a material, a second task of predicting the characteristics of a second material property for a material, and the like. However, it should be understood that the present invention is not limited to a learning method and a prediction method for multi-tasking the relationships between a material and multiple properties, and can be applied to all kinds of tasks that require performing multiple tasks simultaneously, such as relationships between a material and multiple properties.

[0120] Therefore, the domain according to the following examples may mean a category (field) that predicts a certain physical property (e.g., boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity, etc.).

[0121] In an embodiment, such a multi-task learning model (MtLM) can perform pre-training based on certain experimental data.

[0122] Here, the experimental data according to the embodiment refers to learning data used for training a multi-task learning model (MtLM), and may include certain input data and corresponding output data (i.e., label) information.

[0123]

[0124] *In an embodiment, such experimental data may be data including predetermined material-specific characteristic information and corresponding material-specific characteristic information. That is, in an embodiment, the experimental data may be data including predetermined material-specific characteristic information as input data and material property-specific information as output data (i.e., a label) mapped to the input material-specific characteristic information.

[0125] At this time, the material characteristic information according to the embodiment may be information that specifies the unique characteristics possessed by a given material.

[0126] For example, material characteristic information may include at least one of data such as a given material name, molecular structural formula, and / or chemical formula data. In the following description, material characteristic information is limited to molecular structural formula data (e.g., n-dimensional molecular structural formula (n>=2)).

[0127] In addition, the material property specific information according to the embodiment may be information that specifies the data value (i.e., the characteristic value of the property, where the value includes a range) that a given material has for a given material property.

[0128]

[0129] *For example, material property specific information may include property (i.e., domain) values ​​such as boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity of a given material.

[0130] In the following examples, molecular structural formula data representing material-specific characteristic information is used as input data, and physical property value data for each task is used as output data.

[0131] FIG. 6 illustrates an example of a conceptual diagram of a multi-task learning model for predicting multiple property values ​​for a specific material according to one embodiment of the present invention.

[0132] Accordingly, referring to FIG. 19, a multi-tasking learning model (MtLM) according to an embodiment of the present invention may be a multi-tasking model that performs multiple tasks to predict outputs for multiple domains (e.g., multiple physical properties such as boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity) based on predetermined input data (e.g., predetermined molecular structural formula data, etc.).

[0133] Accordingly, the task described above may be to predict multiple domain-specific outputs (i.e., multiple property values) for input data (e.g., predetermined molecular structural formula data, etc.).

[0134] In an embodiment, such a task may include a first task for predicting a first property value for a given material, a second task for predicting a second property value for the material, … and an n-th task for predicting an nth (n>=2) property value for the material, and each of the tasks described above may include a plurality of sub-tasks for predicting property values ​​mapped to each task.

[0135] That is, in the embodiment, the task may include a plurality of sub-tasks for predicting property values ​​mapped to the n-th task, and a main task including the plurality of sub-tasks. In this case, the main task may correspond to the n-th task to which it is mapped.

[0136] In addition, a task according to an embodiment may include a source task, which is a task that provides data to be transferred in a transfer learning process according to an embodiment of the present invention, and a target task, which is a task that receives data to be transferred.

[0137] In other words, in an embodiment, a task can be defined as a source task or a target task depending on whether it corresponds to a subject that transfers data in the learning process or a subject that receives data.

[0138] Returning to the above, in summary, a multi-task learning model (MtLM) that has performed pre-learning according to an embodiment of the present invention can input predetermined material-specific characteristic information and / or material property-specific information, and output predicted data based on the input information and learned knowledge.

[0139] As an example, a multi-task learning model (MtLM) can input certain material-specific characteristic information and output predicted material property-specific information based on the input information and learned knowledge.

[0140] At this time, according to an embodiment, a multi-task learning model (MtLM) can input predetermined material property specific information and output predicted material characteristic information based on the input information and learned knowledge.

[0141] That is, the multi-task learning model (MtLM) may include a model reverse-engineered to predict and output material-specific characteristic information according to predetermined material property-specific information.

[0142] In addition, according to an embodiment, a multi-task learning model (MtLM) can input predetermined material-specific characteristic information and material-specific property information, and output optimal material-specific characteristic information and material-specific property information predicted based on the input information and learned knowledge.

[0143] That is, the multi-task learning model (MtLM) may include a model reverse-engineered to output optimal material-specific characteristic information and material-specific characteristic information predicted based on predetermined material-specific characteristic information and material-specific characteristic information.

[0144] As described above, the multi-tasking learning model (MtLM) according to an embodiment of the present invention learns not only the principles between materials and individual properties but also the relationships between properties and common principles for all learned properties in the process of simultaneously learning tasks for predicting multiple properties for a given material, thereby enabling more accurate prediction of property values ​​and facilitating updates.

[0145] In addition, the multi-task learning model (MtLM) has the advantage of being able to perform learning on a wider range of materials, as the learning data for multiple properties are related to various materials, and thus, the range of materials that can be predicted for each property can be further expanded.

[0146] FIG. 7 illustrates an internal block diagram of a multi-tasking learning model (MtLM) according to one embodiment of the present invention.

[0147] Referring to FIG. 7, in another aspect, a multi-task learning model (MtLM) according to an embodiment may include at least one embedding module (EBM), a task processing unit (TPU), an encoder module (ECM), a regressor module (RGM), a transfer module (TFM), an inverse transfer module (ITM), a perturbation module (PBM), and a loss calculation module (LCM).

[0148] In detail, the embedding module (EBM) according to an embodiment of the present invention may be a pre-encoder module that converts a predetermined input data into an embedding vector.

[0149] Specifically, the embedding module (EBM) compresses high-dimensional data, such as molecular structure data, into a low-dimensional representation, such as an embedding vector, thereby reducing the dimensionality of the input to be processed by the encoder, thereby improving computational efficiency and learning speed, and enabling the encoder to focus on and learn important features in the molecular structure for the pre-learning task.

[0150] Through this, useful features from the model learned for the source task can be easily applied to the model that learns the target task, and generalization is possible for overlapping features between different domains, allowing for effective transfer learning for new domains / tasks.

[0151] In other words, the embedding module (EBM) may be a module that converts specific input data into a vector format by projecting it into a predetermined embedding space.

[0152] As an example, a graph neural network (GNN) suitable for extracting molecular structural features can be used as an embedding module (EBM), and for example, an embedding vector for input data can be provided based on a DMPNN (Directed Message Passing Neural Network) structure.

[0153] FIG. 8 illustrates an example of a conceptual diagram for explaining a multi-tasking learning model (MtLM) including a plurality of task processing units (TPUs) according to one embodiment of the present invention.

[0154] In addition, referring to FIG. 8, a task processing unit (TPU) according to an embodiment of the present invention may be a module that performs a learning and prediction process based on a predetermined task.

[0155] In an embodiment, such a task processing unit (TPU) may include a first task processing unit corresponding to a first domain (e.g., boiling point), a second task processing unit corresponding to a second domain (e.g., melting point), …, an n-th task processing unit corresponding to an n-th domain.

[0156] That is, in the embodiment, the task processing unit (TPU) may include a plurality of first to n-th task processing units (TPUs) corresponding to the number of given domains (i.e., properties).

[0157] At this time, in the embodiment, one of the plurality of task processing units (TPUs) may be a source task processing unit (TPU) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0158] In addition, any one of the remaining task processing units (TPUs) excluding the source task processing unit may be a target task processing unit (TPU) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0159] In detail, a task processing unit (TPU) according to an embodiment may include at least one encoder module (ECM), a regressor module (RGM), a transfer module (TFM), and an inverse module (ITM).

[0160] Specifically, an encoder module (ECM) according to an embodiment of the present invention may be a module that takes a predetermined embedding vector as input and converts the input embedding vector into a latent vector by projecting it into a latent space corresponding to the task.

[0161] In other words, the encoder module (ECM) can be a module that extracts key features of the input embedding vector and expresses them in the corresponding latent space. In detail, the encoder module (ECM) can extract important features from the features of the embedding vector, perform data compression such as removing unnecessary information or noise, and compress the data into a low-dimensional space, and output a latent vector, which is a representation in the latent space.

[0162] In an embodiment, such an encoder module (ECM) may include a plurality of encoder modules (ECMs) corresponding to each of a plurality of domains.

[0163] In an embodiment, the encoder module (ECM) may include a first encoder module corresponding to a first domain (e.g., boiling point), a second encoder module corresponding to a second domain (e.g., melting point), …, an n-th encoder module corresponding to an n-th domain.

[0164] In another embodiment, the encoder module (ECM) may include a third encoder module for performing a first task of predicting solubility in a first solvent in response to a third domain (e.g., solubility), and a fourth encoder module for performing a second task of predicting solubility in a second solvent. That is, another embodiment may include a case of multitasking for different tasks in the same domain. Of course, a multitasking model that integrates multitasking for different domains and multitasking for different tasks in the same domain may also be included in one embodiment of the present invention.

[0165] Below, we will explain based on the fact that different domains represent different tasks.

[0166] At this time, in the embodiment, one of the plurality of encoder modules (ECMs) may be a source encoder module, which is an encoder module (ECM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0167] In addition, any one of the remaining encoder modules (ECMs) excluding the source encoder module may be a target encoder module, which is an encoder module (ECM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0168] In addition, the regressor module (RGM) according to an embodiment of the present invention may be a head module that takes a predetermined latent vector as input and generates a final prediction value according to the input latent vector. That is, in the embodiment, the regressor module (RGM) is used as an example of the head module.

[0169] These regression modules (RGMs) can directly participate in generating the final output and thus determine the model's predictive performance.

[0170] Additionally, in the embodiment, the regressor module (RGM) may include a plurality of regressor modules (RGMs) corresponding to each of a plurality of domains.

[0171] In an embodiment, the regressor module (RGM) may include a first regressor module corresponding to a first domain (e.g., boiling point), a second regressor module corresponding to a second domain (e.g., melting point), …, an n-th regressor module corresponding to an n-th domain.

[0172] At this time, in the embodiment, one of the plurality of regressor modules (RGMs) may be a source regressor module, which is a regressor module (RGM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0173] In addition, any one of the regressor modules (RGMs) other than the source regressor module may be a target regressor module (RGM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0174] In addition, a transfer module (TFM) according to an embodiment of the present invention may be a module that converts a predetermined latent vector into a transfer vector by mapping it to a latent space of another task.

[0175] In detail, in an embodiment, the transfer module (TFM) can transform a specific latent vector into a transfer vector by mapping it to the latent space of another task through the unified latent space (M) based on Riemannian geometry.

[0176] In this process, the transfer module (TFM) can implement geometric alignment between each mapped task according to an embodiment of the present invention.

[0177] That is, in the embodiment, the transfer module (TFM) can implement knowledge transfer by performing geometric alignment between each mapped task during the process of mapping a specific latent vector to the latent space of another task via the unified latent space (M) implemented as a universal local plane (LF). This is described in detail in the multitask model learning method described below.

[0178] That is, in the embodiment, the transfer module (TFM) can effectively perform the transfer of knowledge data between multiple tasks by mapping the latent vector according to the first task to the latent space according to the second task through geometric alignment according to the embodiment of the present invention.

[0179] At this time, in the embodiment, the transfer module (TFM) can support data processing that improves the accuracy and consistency of the transformed vector (i.e., the transition vector) by utilizing an autoencoder structure.

[0180] Additionally, in the embodiment, the transfer module (TFM) may include a plurality of transfer modules (TFMs) corresponding to each of the plurality of domains.

[0181] In an embodiment, the transfer module (TFM) may include a first transfer module corresponding to a first domain (e.g., boiling point), a second transfer module corresponding to a second domain (e.g., melting point), …, an n-th transfer module corresponding to an n-th domain.

[0182] At this time, in the embodiment, one of the plurality of transfer modules (TFMs) may be a source transfer module, which is a transfer module (TFM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0183] In addition, any one of the remaining transfer modules (TFMs) excluding the source transfer module may be a target transfer module, which is a transfer module (TFM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0184] In addition, the inverse module (ITM) according to an embodiment of the present invention may be a module that reconstructs a transition vector mapped and transformed into a latent space of another task by a transfer module (TFM) so that it is mapped back to the original latent space.

[0185] Thus, in the embodiment, the inverse module (ITM) can generate a vector (hereinafter, an inverse vector) that is a reconstructed and transformed vector back to its original state.

[0186] At this time, the inverse module (ITM) in the embodiment can improve the stability of the above-described reconstruction process and the accuracy and consistency of the corresponding transition vector by utilizing the autoencoder structure.

[0187] In an embodiment, such an inverse module (ITM) may include a plurality of inverse modules (ITMs), each corresponding to a plurality of domains.

[0188] In an embodiment, the inverse module (ITM) may include a first inverse module corresponding to a first domain (e.g., boiling point), a second inverse module corresponding to a second domain (e.g., melting point), …, an n-th inverse module corresponding to an n-th domain.

[0189] At this time, in the embodiment, one of the plurality of inverse modules (ITMs) may be a source inverse module, which is an inverse module (ITM) corresponding to a source task of transfer learning according to the embodiment of the present invention.

[0190] In addition, any one of the remaining inverse modules (ITMs) excluding the source inverse module may be a target inverse module (ITM) corresponding to a target task of transfer learning according to an embodiment of the present invention.

[0191] In this way, the multi-task learning model (MtLM) in the embodiment includes multiple task processing units (TPUs) corresponding to each of multiple domains (i.e., properties), thereby forming a latent space for each of multiple tasks performed based thereon.

[0192] And the multi-task learning model (MtLM) can simultaneously learn the transformation in which multiple formed latent spaces are converted into a single integrated latent space (M) according to the pre-learning method described below.

[0193] In other words, the multi-task learning model (MtLM) can perform learning to geometrically align n latent spaces according to various properties into a single unified latent space (M) according to the pre-learning method described below through multiple task processing units (TPUs).

[0194] Therefore, the multi-task learning model (MtLM) can implement simultaneous / batch transfer learning for n*n combinations of properties when n properties are considered.

[0195] Thus, the multi-task learning model (MtLM) according to the embodiment can improve prediction performance through information sharing and learning between interrelated properties / tasks.

[0196] At this time, in the embodiment, the multi-task learning model (MtLM) can implement the transformation between multiple latent spaces and the unified latent space (M) in the same manner regardless of the composition of the property combination (e.g., the first task-second task combination or the second task-third task combination, etc.).

[0197] In other words, the multi-task learning model (MtLM) can perform the transformation between the latent space and the unified latent space (M) between multiple tasks according to various properties in the same way.

[0198] Accordingly, the multi-task learning model (MtLM) can implement data processing that matches geometric characteristics based on latent vectors for multiple latent spaces in the same manner on the unified latent space (M).

[0199] This allows the multitask learning model (MtLM) to effectively maintain consistency in the transformation from each latent space to the unified latent space (M). Consequently, the multitask learning model (MtLM) can more reliably support the flow of mutual information between tasks.

[0200] For example, a multi-task learning model (MtLM) can use information acquired from the first task usefully for the second task as well, if the transformation from the latent space of the first task to the unified latent space (M) and the transformation from the latent space of the second task to the unified latent space (M) are mutually identical.

[0201] Therefore, the multi-task learning model (MtLM) can enhance the sharing of knowledge data between multiple tasks while further improving the model's predictive performance and stability.

[0202] Returning to FIG. 7 again, the perturbation module (PBM) according to an embodiment of the present invention may be a module that generates a plurality of perturbation vectors by applying a predetermined change to a predetermined embedding vector.

[0203] In detail, in an embodiment, the perturbation module (PBM) may be a module that generates a plurality of perturbation vectors (i.e., perturbation points) around a specific embedding vector by applying a change that moves the specific embedding vector in a predetermined direction.

[0204] At this time, the plurality of generated perturbation vectors are designed to maintain a relative distance from the corresponding embedding vector, thereby effectively assisting geometric alignment.

[0205] That is, the above-mentioned perturbation module (PBM) can help align the coordinate systems between the source task and the target task by generating multiple perturbation vectors to assist in the geometric alignment of the model.

[0206] Additionally, in the embodiment, the perturbation module (PBM) can calculate the distance between a predetermined embedding vector and a plurality of perturbation vectors generated based on the embedding vector, and support matching the displacement between the source task and the target task based on the calculated distance.

[0207] This allows the perturbation module (PBM) to more easily maintain consistency in the latent space for the model.

[0208] According to an embodiment, the perturbation module (PBM) can prevent overfitting of the model and improve generalization performance by forcing a relationship between a given embedding vector and multiple perturbation vectors generated based on the embedding vector to be maintained.

[0209] In addition, the loss calculation module (LCM) according to an embodiment of the present invention may be a module that calculates various loss functions based on various vectors obtained through a multi-task learning model (MtLM).

[0210] In an embodiment, the loss calculation module (LCM) may calculate regression loss, autoencoder loss, consistency loss, mapping loss, distance loss, and / or integrated loss according to an embodiment of the present invention. A detailed description thereof will be provided later in the multi-task model learning method.

[0211] This allows the Loss Calculation Module (LCM) to support regularization and learning for different parts of the model, and to provide feedback for model learning, enabling model optimization.

[0212] Meanwhile, in an embodiment of the present invention, the multi-tasking learning model (MtLM) can perform model optimization and update through various data processing processes linked with the modules described above.

[0213] For example, the multi-task learning model (MtLM) can perform model optimization and parameter update in conjunction with the modules described above based on the AdamW optimization algorithm, etc.

[0214] In this way, in the embodiment of the present invention, the multi-tasking learning model (MtLM) not only simultaneously learns knowledge data according to various domains, but also efficiently learns relationships between multiple domains, thereby expanding the learning area and simultaneously performing effective multi-tasking learning that implements batch learning of local patterns according to each domain and common principles between multiple domains.

[0215] Accordingly, the multi-tasking learning model (MtLM) can directly improve the processing performance and accuracy of various multi-tasking tasks based on the model learned as above.

[0216]

[0217] [Multitasking Model Pretraining Method]

[0218] Hereinafter, a method for implementing multi-task model pre-learning in which a computing system (1000) according to one embodiment of the present invention performs transfer learning through geometric alignment in an integrated latent space for multiple tasks according to multiple domains will be described in detail.

[0219] In general, existing transfer learning techniques are mainly focused on classification tasks of image and / or language data sets, and have limitations in solving regression problems or problems in non-Euclidean spaces.

[0220] In particular, when the training data set is insufficient, the decline in prediction performance for the above-described problem is more inevitable, and when multitasking considering various task types is required, the decline in performance is aggravated in learning and prediction for this.

[0221] Additionally, most existing methods are optimized for handling data in Euclidean space, so they do not work effectively in complex curved spaces or nonlinear spaces.

[0222] FIG. 9 illustrates an example of a conceptual diagram for explaining a multi-tasking model pre-learning method according to one embodiment of the present invention.

[0223] Therefore, as shown in FIG. 9, a computing system (1000) according to one embodiment of the present invention seeks to provide a new multi-task model pre-learning method that can overcome the regression problem of a small data set and the limitations of existing transfer learning techniques.

[0224] Hereinafter, in the description according to one embodiment of the present invention, for the sake of effective description, the material described above is limited to a ‘molecule’ and the domain thereof is described based on ‘properties’.

[0225] This is because molecular data sets typically have small amounts of data, contain diverse task types, and primarily deal with regression problems.

[0226] That is, in the case of molecular data sets, various task processing linked to numerous properties is required, but the data provided for this is very limited, and each property has the characteristic of being closely related to or influencing each other.

[0227] Considering these points, the molecular data set is advantageous for application to multi-task processing across multiple domains, and may be a desirable example for explaining a multi-tasking model pre-learning method according to one embodiment of the present invention.

[0228] However, it is not limited thereto, and it is obvious that any embodiment that can apply multi-tasks according to multiple domains can be included in the embodiments of the present invention.

[0229] Hereinafter, a multi-tasking model pre-training method according to one embodiment of the present invention will be described in more detail with reference to the attached drawings.

[0230] FIG. 10 illustrates a block flow diagram for explaining a multi-tasking model pre-training method according to one embodiment of the present invention.

[0231] Referring to FIG. 10, a multi-tasking model pre-training method according to one embodiment of the present invention may include a step (S101) of initializing a multi-tasking learning model (MtLM), a step (S103) of acquiring experimental data, a step (S105) of training the multi-tasking learning model (MtLM) based on the acquired experimental data, and a step (S107) of providing a trained multi-tasking learning model (MtLM).

[0232] In detail, a computing system (1000) according to one embodiment of the present invention can initialize a multi-tasking learning model (MtLM). (S101)

[0233] Here, in other words, the multi-task learning model (MtLM) (Geometrically Aligned Transfer Encoder Model) according to an embodiment of the present invention may be a machine learning model that mutually aligns fragmented knowledge data (for example, latent vectors, etc.) in a latent space for each task through geometric transfer in a single unified latent space (M) in order to process multiple tasks for output according to multiple domains.

[0234] In other words, the multi-tasking learning model (MtLM) according to the embodiment not only simultaneously learns knowledge data according to various domains but also efficiently learns relationships between multiple domains, thereby expanding the learning area and simultaneously performing effective multi-tasking learning that implements batch learning of local patterns according to each domain and common principles between multiple domains.

[0235] In detail, in an embodiment, the computing system (1000) can perform initialization for each component included in the multi-tasking learning model (MtLM) as described above.

[0236] As an example, a computing system (1000) includes an embedding network (MtLM) within a multi-task learning model ( ), encoder network ( ), regressor (head) network ( ), Transfer Network ( ) and / or inverse network( ) etc. random parameters ( ) can be initialized.

[0237] Additionally, as an example, the computing system (1000) can set a predetermined optimization algorithm to be applied to a multi-tasking learning model (MtLM).

[0238] For example, the computing system (1000) may set the AdamW (Decoupled Weight Decay Regularization) algorithm as the optimization algorithm, and according to an embodiment, the optimization algorithm may be improved and used to independently process weight decay.

[0239] Additionally, the computing system (1000) according to one embodiment of the present invention can obtain experimental data. (S103)

[0240] Here, again, experimental data according to an embodiment of the present invention ( ) is learning data used for training a multi-task learning model (MtLM), and may include certain input data and corresponding output data (i.e., label) information.

[0241] In an embodiment, such experimental data may be data including predetermined material-specific characteristic information and corresponding material-specific characteristic information. That is, in an embodiment, the experimental data may be data including predetermined material-specific characteristic information as input data and material property-specific information as output data (i.e., a label) mapped to the input material-specific characteristic information.

[0242] At this time, the material characteristic information according to the embodiment may be information specifying the unique characteristics possessed by a given material. That is, the material characteristic information in the embodiment may be information specifying the unique characteristics possessed by a given molecule.

[0243] For example, material characteristic information may include at least one or more data from among a given material name, molecular structural formula, and / or chemical formula data, and in the following description, material characteristic information is limited to molecular structural formula data (e.g., n (n>=2)-dimensional molecular structural formula) data.

[0244] In addition, the material property specific information according to the embodiment may be information that specifies the data value (i.e., the characteristic value of the property, where the value includes a range) that a given material has for a given material property.

[0245] For example, material property specific information may include property (i.e., domain) values ​​such as boiling point, melting point, refractive index, solubility, viscosity, surface tension, density, strength, and / or thermal conductivity of a given material.

[0246] In other words, in the following examples, molecular structural formula data representing material-specific characteristic information is used as input data, and physical property value data for each task is used as output data.

[0247] In detail, in an embodiment, the computing system (1000) can obtain experimental data as described above based on predetermined user input and / or connection with an external server.

[0248] In more detail, in an embodiment, the computing system (1000) can obtain property relationship data indicating a relationship between certain properties.

[0249] Specifically, in the embodiment, the computing system (1000) can obtain the property relationship data based on user input (e.g., input of data manually collected by the user, etc.) and / or linkage with a predetermined artificial intelligence model.

[0250] As an example, a computing system (1000) can obtain property relationship data based on prompt engineering by linking with a large language model that has performed a predetermined pre-learning.

[0251] In detail, the computing system (1000) can perform a data search based on keywords for physical properties in a database of specialized data (e.g., papers, patents, and / or academic materials) through a specific large language model.

[0252] Additionally, the computing system (1000) can extract at least one piece of information indicating a relationship between certain properties from the retrieved data.

[0253] And the computing system (1000) can perform an editing process to organize and classify the extracted property relationship information according to criteria such as property type and / or relationship type.

[0254] Thus, the computing system (1000) can obtain property relationship data according to the edited information.

[0255] Additionally, in the embodiment, the computing system (1000) can create a database of acquired material property relationship data.

[0256] In detail, the computing system (1000) according to the embodiment may include a material property relationship data database.

[0257] Here, the above-described property relationship data database can store data including information on relationships between certain properties.

[0258] These property relationship data databases can store data collected manually by people, or data automatically searched, extracted, and edited using pre-trained artificial intelligence models.

[0259] That is, in the embodiment, the computing system (1000) can store and manage the material property relationship data obtained as above in a material property relationship data database.

[0260] Thus, the computing system (1000) can build a property relationship data database that includes information on the relationship between various properties.

[0261] Thereafter, in the embodiment, the computing system (1000) can obtain property relationship information that specifies the relationship between different properties based on the property relationship data included in the property relationship data database.

[0262] As an example, a computing system (1000) can obtain at least one piece of property relationship information according to a predetermined property relationship data by linking with a large language model that has performed a predetermined pre-learning.

[0263] Figure 11 illustrates an example of a knowledge graph representing relationships between properties according to one embodiment of the present invention.

[0264] For example, referring to FIG. 11, the computing system (1000) can obtain a predetermined knowledge graph as property relationship information, and at this time, the knowledge graph can include relationship information between the first to n-th properties, including relationship information between the first property (P1) and the second property (P2), relationship information between the first property (P1) and the third property (P3), etc.

[0265] As a specific example, the property relationship information as described above may include information about properties associated with a specific property, information about the properties of the relationship (e.g., conflict, similarity, correlation, cause and effect, independence, proportionality, and / or inverse relationship), and / or information about the degree of association (association strength) according to the properties of the determined relationship.

[0266] Here, the properties of the relationship (hereinafter, property relationship properties) and the degree of association information according to one embodiment may be displayed on the knowledge graph by being divided into a number indicating the association in a positive correlation and a number indicating the association in a negative correlation.

[0267] Additionally, in the embodiment, the computing system (1000) can obtain the experimental data described above based on the obtained property relationship information.

[0268] That is, in the embodiment, the computing system (1000) can obtain experimental data including predetermined material characteristic information and corresponding material characteristic information based on various material property relationship information obtained as described above.

[0269] As an example, a computing system (1000) can obtain at least one experimental data according to predetermined property relationship information by linking with a large language model that has performed predetermined pre-learning.

[0270] In the above, it has been described that the computing system (1000) acquires experimental data through a predetermined data preprocessing based on the given material property relationship information, but this is only an example, and various embodiments are possible, such as using the given material property relationship information itself as experimental data, depending on the embodiment.

[0271] As a specific example, the computing system (1000) can obtain first experimental data including information on a first property for the molecular structures of a plurality of substances, second experimental data including information on a second property for the molecular structures of a plurality of substances, etc.

[0272] Here, the plurality of substances corresponding to the first experimental data described above and the plurality of substances corresponding to the second experimental data may be different from each other or at least partially the same.

[0273] In this way, in the embodiment, the computing system (1000) can acquire the aforementioned experimental data based on information on the relationship between properties collected from data sources of various origins, thereby implementing model learning based on richer learning data and minimizing problems of errors and data insufficiency due to data bias.

[0274] Accordingly, the computing system (1000) can support a learned model to perform more accurate and reliable predictions by maximizing the effect of various data sources, and can directly improve multi-task processing and prediction performance according to multiple properties.

[0275] At this time, in the embodiment of the present invention, the computing system (1000) can reflect the obtained property relationship information (e.g., degree of correlation information, etc.) in the weight of at least one loss among various losses used in training the multi-task learning model (MtLM) in step S205 described below.

[0276] That is, in the embodiment, the computing system (1000) can perform effective multi-tasking learning that implements batch learning of local patterns according to each domain (i.e., properties) and common principles between multiple domains by reflecting property relationship information that specifies the relationship between various properties.

[0277] Meanwhile, according to an embodiment, the computing system (1000) can detect, if a relationship between a predetermined first property and a second property is expressed as a formula in a property relationship data database.

[0278] And the computing system (1000) can augment an experimental data set for training a multi-task learning model (MtLM) using the detected formula (Data augmentation).

[0279] For example, the computing system (1000) can apply data augmentation based on a formula detected from a data set in which only the first property exists for a given material to expand the data set to include the second property.

[0280] Alternatively, according to an embodiment, the computing system (1000) may optimize and update a multi-task learning model (MtLM) using the formula detected as above.

[0281] For example, the computing system (1000) may update the multi-tasking learning model (MtLM) to predict the second property by applying the detected formula to the first property when the multi-tasking learning model (MtLM) has learned only the first property among the first and second properties.

[0282] Alternatively, according to an embodiment, the computing system (1000) can improve the accuracy of integrated latent space (M) mapping during transfer learning for a multi-task learning model (MtLM) using the detected formula.

[0283] For example, if a multi-task learning model (MtLM) has learned both tasks of predicting a first material property and a second material property, the computing system (1000) can re-perform transfer learning between the model of the task of predicting the first material property and the model of the task of predicting the second material property based on the detected formula. Accordingly, the computing system (1000) can enhance the integrated latent space (M) mapping optimization of the multi-task learning model (MtLM), thereby improving its accuracy.

[0284] In addition, a computing system (1000) according to one embodiment of the present invention can train a multi-task learning model (MtLM) based on acquired experimental data. (S105)

[0285] FIG. 12 illustrates a block flow diagram for explaining a multi-tasking learning model (MtLM) training method according to one embodiment of the present invention, FIG. 13 illustrates an example of a first conceptual diagram for explaining a multi-tasking learning model (MtLM) training method according to one embodiment of the present invention, and FIG. 14 illustrates an example of a second conceptual diagram for explaining a multi-tasking learning model (MtLM) training method according to one embodiment of the present invention.

[0286] That is, referring to FIGS. 12 to 14, in the embodiment, the computing system (1000) can perform pre-learning for a multi-task learning model (MtLM) based on the experimental data acquired as described above.

[0287] At this time, in the embodiment, the computing system (1000) can simultaneously train multiple tasks that predict at least two domains (i.e., material properties) for a multi-task learning model (MtLM).

[0288] In detail, as an example, a computing system (1000) can collectively perform transfer learning-based pre-training on the source task and the target task, when a first task performing prediction on a first property is referred to as a source task and an n-th task performing prediction on an n-th (n>=2) property is referred to as a target task, and vice versa.

[0289] In other words, in the embodiment, the computing system (1000) can simultaneously perform multiple transfer learning according to n*n combinations between the first to nth properties based on a multi-task learning model (MtLM) when considering n properties.

[0290] Therefore, the computing system (1000) according to an embodiment of the present invention can make predictions for each property more accurately by allowing a multi-tasking learning model (MtLM) to learn a plurality of properties while also naturally learning correlations between the properties learned together through transfer learning.

[0291] As a specific example, the computing system (1000) can pre-train a multi-task learning model (MtLM) on the relationships between the first to third properties based on experimental data containing relationship information between the first to third properties. Thereafter, the computing system (1000) can more accurately predict the relationship information between the first to third properties for a first material that only has relationship information on the first and second properties by using the pre-trained multi-task learning model (MtLM) as described above.

[0292] In addition, through this, the computing system (1000) can implement task processing that operates robustly to deformation of molecular structures and provides accurate prediction values ​​for properties by training a multi-task learning model (MtLM) for various molecular structure types through transfer learning when experimental data for each property includes data of different molecular structure types.

[0293] That is, the computing system (1000) according to the embodiment can perform pre-learning to batch-learn not only various material property data but also relationships between multiple material properties to a multi-tasking learning model (MtLM), thereby expanding the model learning area and simultaneously implementing effective multi-tasking learning that batch-learns local patterns according to each material property and common principles between multiple material properties.

[0294] Accordingly, the computing system (1000) can significantly improve the performance and quality of processing various multitasking tasks based on the pre-learned multitasking learning model (MtLM) as described above, thereby expanding the range in which accurate prediction is possible.

[0295] Returning to the example, the computing system (1000) can perform pre-learning based on transfer learning as described above according to the following process.

[0296] In detail, in an embodiment, the computing system (1000) can set a training loop for a multi-task learning model (MtLM). (S201)

[0297] In more detail, in an embodiment, the computing system (1000) can set the number of epoch repetitions, the number of task repetitions, and / or the number of batch repetitions during training.

[0298] As an example, the computing system (1000) can set a training loop to repeatedly perform epoch 'i' from '1 to n (n>=1)' during training, repeatedly perform the same for each task 't', and repeatedly perform the same for each preset batch 'b'.

[0299] Additionally, in the embodiment, the computing system (1000) can obtain a geometric alignment vector based on the experimental data obtained as described above. (S203)

[0300] Here, the geometric alignment vector according to the embodiment of the present invention may mean various vectors obtained through a multi-task learning model (MtLM).

[0301] In the embodiment, the geometric alignment vector is an embedding vector ( ), pulturation vector ( ), encoding vectors, transition vectors, and inverse vectors.

[0302] In detail, in an embodiment, the computing system (1000) can input the acquired experimental data into a multi-task learning model (MtLM).

[0303] Additionally, in the embodiment, the computing system (1000) can obtain an embedding vector based on a multi-task learning model (MtLM) that inputs experimental data.

[0304] In more detail, the computing system (1000) can convert the input experimental data into an embedding vector through an embedding network in conjunction with the embedding module (EBM) of the multi-task learning model (MtLM).

[0305] Accordingly, the computing system (1000) can obtain an embedding vector converted into a vector format by projecting the experimental data into a predetermined embedding space.

[0306] Additionally, in the embodiment, the computing system (1000) can generate a perturbation vector based on the acquired embedding vector.

[0307] In detail, in an embodiment, the computing system (1000) can generate a plurality of perturbation vectors (i.e., perturbation points) on a predetermined periphery based on the obtained embedding vector in conjunction with the perturbation module (PBM) of the multi-task learning model (MtLM).

[0308] At this time, in the embodiment, the computing system (1000) can obtain a corresponding perturbation vector for each task by repeatedly performing the above-described functional operation for each task.

[0309] As an example, the computing system (1000) can obtain a plurality of task-specific fusion vectors, including a fusion vector corresponding to task 't' and a fusion vector corresponding to task 's'.

[0310] Additionally, in the embodiment, the computing system (1000) can obtain encoding vectors, transition vectors, and inverse vectors based on the generated perturbation vectors and embedding vectors.

[0311] In detail, referring further to FIGS. 8, 13 and 14, in an embodiment, the computing system (1000) can obtain an encoding vector, a transition vector and an inverse vector for each task processing unit (TPU) by collectively interworking with a plurality of task processing units (TPUs) included in a multi-tasking learning model (MtLM).

[0312] That is, the computing system (1000) can obtain a plurality of encoding vectors, transition vectors, and inverse vectors for each task processed in each task processing unit (TPU) by collectively linking with a plurality of task processing units (TPUs) each corresponding to a plurality of properties (i.e., domains) to be predicted.

[0313] Hereinafter, for the sake of effective explanation, a method for obtaining the vectors is described based on a task 't' processed in a first task processing unit (TPU) and a task 's' processed in a second task processing unit (TPU). However, the plurality of task processing units (TPUs) described above can all obtain the vectors for each task corresponding to each unit in the same manner as the method described below.

[0314] In detail, in the embodiment, the computing system (1000) can obtain an encoding vector based on the generated perturbation vector and embedding vector.

[0315] Here, the encoding vector according to the embodiment may include a pultering latent vector, which is a latent vector generated based on a predetermined pultering vector, and an original latent vector generated based on an embedding vector, which is an original vector of the pultering vector.

[0316] In more detail, in an embodiment, the computing system (1000) can project the generated perturbation vector into a latent space corresponding to the task through an encoder network by linking with an encoder module (ECM) included in the first and second task processing units (TPUs) (hereinafter, training task processing units) of the multi-task learning model (MtLM) to convert the generated perturbation vector into a latent vector.

[0317] Additionally, in the embodiment, the computing system (1000) can convert the obtained embedding vector into a latent vector by projecting it into a latent space corresponding to the task through an encoder network in conjunction with an encoder module (ECM) of a multi-task learning model (MtLM).

[0318] Thus, in the embodiment, the computing system (1000) can obtain a perturbation potential vector and an original potential vector.

[0319] At this time, in the embodiment, the computing system (1000) can repeatedly perform the above-described functional operation for each task to obtain the corresponding original potential vector and the perturbation potential vector for each task.

[0320] As an example, the computing system (1000) generates an original latent vector ( : Hereinafter, the original latent vector of the task 't' and the pulverization latent vector corresponding to the task 't' ( : Below, the jet pulverization potential vector can be obtained.

[0321] Additionally, the computing system (1000) generates an original latent vector ( : Below, the original latent vector of the task 's' and the pulverization latent vector corresponding to the task 's' ( : Below, you can obtain the potential vector of the s-pulteration.

[0322] Additionally, in the embodiment, the computing system (1000) can obtain a transition vector based on the obtained encoding vector.

[0323] Here, the transition vector according to the embodiment may include a pulverization transfer vector, which is a transition vector generated based on a predetermined pulverization potential vector, and an original transfer vector, which is a transition vector generated based on an original potential vector corresponding to the pulverization potential vector.

[0324] In detail, in an embodiment, the computing system (1000) can convert the acquired perturbation latent vector and original latent vector into a transition vector by mapping them to the latent space of another task (in an embodiment, task 's' or task 't') through a transfer network in conjunction with a transfer module (TFM) included in a training task processing unit of a multi-task learning model (MtLM).

[0325] Thus, the computing system (1000) can obtain a perturbation transfer vector and an original transfer vector.

[0326] At this time, in the embodiment, the computing system (1000) can repeatedly perform the above-described functional operation for each task to obtain the corresponding original transfer vector and perturbation transfer vector for each task.

[0327] As an example, a computing system (1000) may be configured to provide an original transfer vector ( : Below, the original transfer vector t) and the perturbation transfer vector corresponding to the task 't' ( : Below, the jet pulverization transfer vector can be obtained.

[0328] Additionally, the computing system (1000) provides an original transfer vector ( : Below, the original transfer vector of s) and the transformation transfer vector corresponding to task 's' ( : Below, you can obtain the s perturbation transfer vector.

[0329] Thus, in the embodiment, the computing system (1000) can obtain geometric alignment vectors (i.e., embedding vectors, perturbation vectors, encoding vectors (including original latent vectors and perturbation latent vectors), and transfer vectors (including original transfer vectors and perturbation transfer vectors)) based on experimental data.

[0330] Additionally, in the embodiment, the computing system (1000) can obtain an inverse vector based on the acquired transition vector.

[0331] Here, the inverse vector according to the embodiment may include a pulter inverse vector, which is an inverse vector generated based on a predetermined pulter transfer vector, and an original inverse vector, which is an inverse vector generated based on an original transfer vector corresponding to the pulter transfer vector.

[0332] In detail, in an embodiment, the computing system (1000) can reconstruct the acquired perturbation transfer vector and the original transfer vector through an inverse network so that they are mapped back to the original latent space and converted into inverse vectors in conjunction with an inverse module (ITM) included in the training task processing unit of the multi-task learning model (MtLM).

[0333] Thus, the computing system (1000) can obtain the perturbation inverse vector and the original inverse vector.

[0334] At this time, in the embodiment, the computing system (1000) can repeatedly perform the above-described functional operation for each task to obtain the corresponding original inverse vector and perturbation inverse vector for each task.

[0335] As an example, the computing system (1000) generates an original inverse vector ( : Below, the original inverse vector of the t and the perturbation inverse vector corresponding to the task 't' ( : Below, the jet t perturbation inverse vector can be obtained.

[0336] Additionally, the computing system (1000) generates the original inverse vector ( : Below, the original inverse vector of the task 's' and the perturbation inverse vector corresponding to the task 's' ( : Below, you can obtain the s-pulteration inverse vector.

[0337] Thus, in the embodiment, the computing system (1000) can obtain geometric alignment vectors based on experimental data (i.e., embedding vectors, perturbation vectors, encoding vectors (including original latent vectors and perturbation latent vectors), transition vectors (including original transfer vectors and perturbation transfer vectors), and inverse vectors (including original inverse vectors and perturbation inverse vectors), etc.).

[0338] Additionally, in the embodiment, the computing system (1000) can calculate a geometric alignment loss based on the obtained geometric alignment vector. (S205)

[0339] Here, the geometric alignment loss according to the embodiment of the present invention may mean various loss functions (Loss) calculated based on various vectors (i.e., geometric alignment vectors) obtained through a multi-task learning model (MtLM).

[0340] In the embodiment, the geometric alignment loss is the regression loss ( : Regression loss), autoencoder loss ( : Autoencoder loss), consistency loss( : Consistency loss), mapping loss( : Mapping loss), distance loss( : Distance loss) and / or integration loss( : Integrated loss) etc.

[0341] In detail, in the embodiment, the computing system (1000) can calculate a geometric alignment loss based on the geometric alignment vector obtained as described above (i.e., the geometric alignment vector for each of the plurality of task processing units (TPUs) and the plurality of tasks processed by the task processing units (TPUs).

[0342] As mentioned above, for the sake of effective explanation, a method for calculating the geometric alignment loss is described below based on task 't' processed in the first task processing unit (TPU) and task 's' processed in the second task processing unit (TPU) (especially, with more emphasis on task 't').

[0343] At this time, in the embodiment, the computing system (1000) can train a multi-task learning model (MtLM) by reflecting the property relationship information described in the aforementioned step S103 in the weight for at least one loss among various losses included in the geometric alignment loss.

[0344] FIG. 15 and FIG. 16 illustrate examples of drawings for explaining a method for calculating regression loss according to one embodiment of the present invention.

[0345] In more detail, referring to FIGS. 14 to 16, in an embodiment, a computing system (1000) can calculate a regression loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0346] In more detail, in the embodiment, the computing system (1000) predicts a predicted value () through a regression module (RGM) according to the following [Mathematical Formula 1]. ) and actual value( , i.e., the regression loss based on the label value) can be calculated. Here, the predicted value of [Mathematical Formula 1] is ' ' can also be expressed as.

[0347] [Mathematical Formula 1]

[0348]

[0349] That is, the computing system (1000) can calculate the regression loss by calculating the mean squared error (MSE) between the predicted value and the actual value.

[0350] At this time, in the embodiment, each task can prevent mutual interference by calculating an independent regression loss based on an encoder module (ECM) and a regressor module (RGM) matching each task and performing learning based on the regression loss.

[0351] In this way, the computing system (1000) can easily evaluate the regression performance of the model by calculating the regression loss.

[0352] In addition, referring further to FIG. 14, in the embodiment, the computing system (1000) can calculate the autoencoder loss2) based on a multi-task learning model (MtLM) that has obtained a geometric alignment vector.

[0353] In detail, in an embodiment, the computing system (1000) can calculate an autoencoder loss based on the original latent vector and the original inverse vector according to the following [Mathematical Formula 2].

[0354] [Mathematical Formula 2]

[0355]

[0356] That is, the computing system (1000) can calculate the autoencoder loss by calculating the mean square error (MSE) between the latent vector and the inverse vector.

[0357] In an embodiment, the computing system (1000) can improve accuracy in the data transfer process through the autoencoder loss calculated as above.

[0358] FIG. 17 illustrates an example of a drawing for explaining an integrated latent space (M) mapping method according to one embodiment of the present invention.

[0359] Meanwhile, referring to FIG. 17, in an embodiment, the computing system (1000) can learn a bidirectional transformation matrix (TM) that can be mapped to a common integrated latent space (M) for each task.

[0360] In detail, in an embodiment, the computing system (1000) can connect latent spaces between tasks by utilizing knowledge data that contain labels for both tasks.

[0361] In this process, the computing system (1000) can calculate consistency loss and mapping loss according to the embodiment.

[0362] FIG. 18 and FIG. 19 illustrate examples of drawings for explaining a consistency loss calculation method according to one embodiment of the present invention.

[0363] In more detail, referring to FIGS. 14, 18 and 19, in an embodiment, the computing system (1000) can calculate consistency loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0364] Specifically, in the embodiment, the computing system (1000) can calculate the consistency loss based on the perturbation transfer vector of task 't' and the perturbation transfer vector of task 's' according to [Mathematical Formula 3] below.

[0365] [Equation 3]

[0366]

[0367] That is, the computing system (1000) can calculate the consistency loss by calculating the mean square error (MSE) between the t-pulter transfer vector and the s-pulter transfer vector.

[0368] At this time, in the embodiment, the computing system (1000) can derive a metric for calculating a distance in space from a transformation matrix (TM), and learn to make the distance in the latent space of each task the same based on the derived metric.

[0369] Through this, the computing system (1000) can more effectively implement geometric alignment between tasks.

[0370] FIG. 20 and FIG. 21 illustrate examples of drawings for explaining a mapping loss calculation method according to one embodiment of the present invention.

[0371] In addition, referring to FIGS. 14, 20 and 21, in the embodiment, the computing system (1000) can calculate a mapping loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0372] In detail, in the embodiment, the computing system (1000) can calculate a mapping loss based on a predicted value based on an actual value according to task 't' and an original inverse vector according to task 's' according to [Mathematical Formula 4] below.

[0373] [Equation 4]

[0374]

[0375] That is, the computing system (1000) can calculate the mapping loss by calculating the mean square error (MSE) between the actual value of task 't' and the predicted value according to the original inverse vector of task 's'.

[0376] In an embodiment, the computing system (1000) can implement learning to transfer latent vectors from the latent space of one task to the latent space of the other task by calculating the mapping loss as described above, and perform the other task based on the transferred vectors, thereby inducing latent characteristics to become similar to each other.

[0377] Through this, the computing system (1000) can evaluate the prediction performance of vectors transferred to the latent space of other tasks and induce learning in a direction to improve it.

[0378] In addition, referring further to FIG. 14, in the embodiment, the computing system (1000) can calculate distance loss5) based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0379] In detail, in the embodiment, the computing system (1000) calculates the distance between the original transfer vector and the perturbation transfer vector of each task according to [Mathematical Equation 5] and [Mathematical Equation 6] below. : The distance loss between tasks can be calculated based on the transfer vector displacement.

[0380] In more detail, in the embodiment, the computing system (1000) calculates the distance between the t original transfer vector and the t perturbation transfer vector according to the task 't' according to [(a) of the mathematical expression 5] below. : Below, the t transfer vector displacement) can be calculated.

[0381] In addition, the computing system (1000) calculates the distance between the s original transfer vector and the s perturbation transfer vector according to the task 's' according to [(b) of the mathematical expression 5] below. : Below, the transfer vector displacement) can be calculated.

[0382] [Equation 5]

[0383]

[0384] Additionally, in the embodiment, the computing system (1000) can calculate the mean square error (MSE) between the t transfer vector displacement and the s transfer vector displacement according to the following [Mathematical Formula 6] to calculate the distance loss.

[0385] [Equation 6]

[0386]

[0387] Here, 'M' in [Mathematical Formula 6] means the number of pulverization points.

[0388] At this time, in the embodiment, the computing system (1000) can define the t transfer vector displacement and the s transfer vector displacement as displacements in the source task and the target task, respectively.

[0389] Thus, the computing system (1000) can more easily calculate the distance between the original transfer vector and the perturbation transfer vector by interpreting the t transfer vector displacement and the s transfer vector displacement as being in a flat Euclidean space.

[0390] Therefore, the computing system (1000) can support more complete consistency maintenance of the latent space of the model.

[0391] FIG. 22 illustrates an example of a drawing for explaining an integrated loss calculation method according to one embodiment of the present invention.

[0392] In addition, referring to FIG. 14 and FIG. 22, in the embodiment, the computing system (1000) can calculate an integrated loss based on a multi-task learning model (MtLM) that has acquired a geometric alignment vector.

[0393] In detail, in the embodiment, the computing system (1000) can calculate an integrated loss by weighting the regression loss, autoencoder loss, consistency loss, mapping loss, and distance loss described above according to [Mathematical Formula 7] below.

[0394] [Equation 7]

[0395]

[0396] At this time, in the embodiment, the computing system (1000) can apply weights to each loss function so that each loss function can be optimized for a specific aspect of the model.

[0397] Here, ' in [Equation 7] ' is the weight of the autoencoder loss, and ' ' is the weight of consistency loss, and ' ' is the weight of the mapping loss, and ' ' is the weight of the distance loss.

[0398] In an embodiment, the computing system (1000) can update parameters in a direction to minimize the integrated loss by adjusting the importance of the loss function corresponding to each weight during the learning process of the model by utilizing the above weights.

[0399] At this time, in the embodiment, the computing system (1000) can adjust the weight for at least one loss of the integrated loss according to the property relationship information obtained in the aforementioned step S103.

[0400] As an example, the computing system (1000) can determine or correct the value of the weight of the mapping loss, which is a loss that supports learning of relationships between properties, by reflecting the above-described property relationship information.

[0401] As a specific example, the computing system (1000) can reflect property relationship information in a manner that increases the weight of the mapping loss and strengthens learning according to the mapping loss when a predetermined first property and a second property are correlated and have a high degree of mutual correlation.

[0402] As another example, the computing system (1000) can reflect the property relationship information in a way that reduces the weight of the mapping loss and weakens learning according to the mapping loss when the first property and the second property are not correlated or have a low degree of correlation.

[0403] Accordingly, the computing system (1000) can more accurately apply the relationship information between properties detected from various sources of previously studied expert data to multi-task learning model (MtLM) training.

[0404] Returning to Figure 12, the computing system (1000) in the embodiment can also perform model optimization and parameter update based on the geometric alignment loss calculated as described above. (S207)

[0405] In detail, in an embodiment, the computing system (1000) can perform optimization and parameter update for a multi-task learning model (MtLM) based on the integrated loss described above.

[0406] As an example, the computing system (1000) can calculate a gradient based on the integrated loss for each parameter of a multi-task learning model (MtLM) through backpropagation.

[0407] And the computing system (1000) can perform parameter update of a multi-task learning model (MtLM) using a calculated gradient and a preset optimization algorithm (e.g., AdamW (Decoupled Weight Decay Regularization) algorithm, etc.).

[0408] Thus, the computing system (1000) can implement a multi-task learning model (MtLM) optimization based on geometric alignment loss (particularly, integration loss).

[0409] In this way, in the embodiment, the computing system (1000) can perform multi-task learning model (MtLM) optimization and parameter update learning through a combination of multiple loss functions calculated in various ways.

[0410] At this time, each loss function can easily assist in improving the performance of the model by correcting the accuracy, consistency, and / or distance of the knowledge data mapping.

[0411] Through this, the computing system (1000) can implement a multi-tasking model that provides improved performance that overcomes the regression problem of a small data set and the limitations of existing transfer learning techniques, while operating more stably and providing improved generalization performance.

[0412] Additionally, in an embodiment, the computing system (1000) may terminate multi-task learning model (MtLM) training. (S209)

[0413] In detail, in an embodiment, the computing system (1000) can terminate the multi-task learning model (MtLM) training process described above when a preset training termination condition is met.

[0414] In an embodiment, the computing system (1000) can terminate multi-task learning model (MtLM) training upon completion of a set training loop.

[0415] In this way, in the embodiment, the computing system (1000) can simultaneously train a multi-tasking learning model (MtLM) to predict at least two properties, thereby allowing the multi-tasking learning model (MtLM) to naturally learn correlations between properties learned together while learning multiple properties through transfer learning, thereby making predictions for each property more accurate.

[0416] In addition, through this, the computing system (1000) can implement task processing that operates robustly to deformation of molecular structures and provides accurate prediction values ​​for properties by training a multi-task learning model (MtLM) for various molecular structure types through transfer learning when experimental data for each property includes data of different molecular structure types.

[0417] That is, the computing system (1000) according to the embodiment can perform pre-learning to batch-learn not only various material property data but also relationships between multiple material properties to a multi-tasking learning model (MtLM), thereby expanding the model learning area and simultaneously implementing effective multi-tasking learning that batch-learns local patterns according to each material property and common principles between multiple material properties.

[0418] Accordingly, the computing system (1000) can significantly improve the performance and quality of processing various multitasking tasks based on the pre-learned multitasking learning model (MtLM) as described above, thereby expanding the range in which accurate prediction is possible.

[0419] Returning to Figure 10, a computing system (1000) according to an embodiment of the present invention can also provide a trained multi-task learning model (MtLM). (S107)

[0420] That is, in the embodiment, the computing system (1000) can provide a multi-task learning model (MtLM) trained as described above in a predetermined manner.

[0421] In an embodiment, the computing system (1000) may provide a multi-task learning model (MtLM) trained according to an embodiment of the present invention in conjunction with a predetermined application service (e.g., a material synthesis / evaluation service, a material property prediction service, and / or an optimal material recommendation service, etc.).

[0422] Specifically, according to an embodiment, the computing system (1000) may provide a multi-tasking learning model (MtLM) through a service that inputs a predetermined molecular structural formula into the multi-tasking learning model (MtLM) and outputs property values ​​for a plurality of domains (i.e., properties) that the multi-tasking learning model (MtLM) has pre-learned. At this time, each property value may be provided in the form of a most probable property value and / or a property value range for a specific probability.

[0423] Conversely, according to an embodiment, the computing system (1000) may provide a multi-tasking learning model (MtLM) by reverse engineering a pre-learned multi-tasking learning model (MtLM), and, when a plurality of property values ​​are input, inputting the property values ​​into the reverse-engineered multi-tasking learning model (MtLM) and outputting at least one molecular structural formula that satisfies the property values.

[0424] In this way, the computing system (1000) in the embodiment can effectively support processing of various multitasking tasks in various ways by using a multitasking learning model (MtLM) with improved performance according to the embodiment of the present invention.

[0425] As described above, in the embodiment of the present invention, the computing system (1000) can provide a multi-task learning model (MtLM) that provides improved performance that overcomes the regression problem of a small data set and the limitations of existing transfer learning techniques by mutually transferring and learning knowledge data of latent spaces for each task through geometric alignment in a single integrated latent space in order to process multiple tasks for output according to multiple domains, and at the same time operates more stably.

[0426] Through this, the computing system (1000) can provide a transfer learning-based multi-tasking model that operates stably and robustly while exhibiting high generalization performance even in situations where the amount of given data is small, various task types are included, or regression problems are mainly dealt with.

[0427] In other words, the computing system (1000) can provide a multi-task learning model (MtLM) with improved prediction performance based on knowledge distilled through geometric alignment-based transfer learning performed in conjunction with other domains, even if there is a domain among multiple domains (in the embodiment, physical properties) that lacks experimental data (learning data).

[0428] For example, the computing system (1000) pre-trains a multi-task learning model (MtLM) based on the first to tenth properties for each of a plurality of molecular structural formulas, and then, when a first molecular structural formula including only data for the first to fifth properties is input, the computing system (1000) can more accurately predict the predicted values ​​for the remaining sixth to tenth properties for the first molecular structural formula based on the knowledge data transferred and distilled through pre-training, and generate and provide output data based thereon.

[0429] In this way, the computing system (1000) according to an embodiment of the present invention can provide a multi-tasking model that implements effective transfer learning based on geometric alignment, guarantees high generalization performance, improves prediction accuracy for regression problems, supports regularization according to a combination of various loss functions, and performs a stable learning process to guarantee robust performance.

[0430] Above, the method and system for learning a multi-tasking model using a data-centric technique according to one embodiment of the present invention have the effect of accurately predicting an integrated output that satisfies each requirement of a plurality of domains by performing transfer learning through geometric alignment in an integrated latent space for multiple tasks according to a plurality of domains.

[0431] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention can simultaneously train various prediction tasks according to multiple domains in the transfer learning process, thereby learning not only individual principles of each domain but also correlations between domains and common principles for the entire domain, thereby expanding the model learning area and simultaneously expanding the prediction acceptance range for each domain and domain.

[0432] Accordingly, the multitasking model learning method and system using a data-centric technique according to one embodiment of the present invention have the effect of directly improving the performance and quality of processing various multitasking tasks using the learned model.

[0433] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of easily supporting the transfer of knowledge between interrelated data and the improvement of prediction performance accordingly by implementing the exchange of mutual information by matching geometric characteristics between the various prediction tasks.

[0434] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of increasing resistance to unnecessary interference information and increasing the stability of the model.

[0435] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention can increase the diversity of data used for model learning and improve learning performance by enabling the model to learn more information by securing a data set for training using source data from various sources.

[0436] In addition, the method and system for learning a multi-tasking model using a data-centric technique according to an embodiment of the present invention can predict multiple properties for a specific substance by applying the learned multi-tasking learning model (MtLM) to predict relationships between multiple properties and materials, and can provide a multi-tasking model capable of predicting a specific substance satisfying multiple properties, thereby providing a multi-tasking model that can be universally utilized for various substances (materials), thereby having the effect of improving the quality of the entire related industry.

[0437] A method and system for learning a multi-tasking model using a data-centric technique according to one embodiment of the present invention can provide a multi-tasking model that maintains high performance for multiple tasks even on a small data set by transferring knowledge learned from a source task to a target task through transfer learning to resolve the problem of insufficient data.

[0438] Therefore, the multi-tasking model learning method and system using a data-centric technique according to one embodiment of the present invention have the effect of expanding the scope of application to fields where it was difficult to apply machine learning models due to insufficient data or domain knowledge.

[0439] In addition, the multi-task model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of demonstrating high prediction performance even in complex regression problems such as molecular data sets by providing a specialized transfer learning technique that can be effectively applied to regression problems.

[0440] In addition, the multi-tasking model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of improving the efficiency of transfer learning by maintaining geometric consistency between tasks by optimizing knowledge transfer between source tasks and target tasks through a Riemannian geometric approach.

[0441] In addition, the multi-task model learning method and system according to the data-centric technique according to one embodiment of the present invention have the effect of further improving the generalization performance of the model by combining multiple loss functions to normalize various aspects of the model.

[0442]

[0443] [Method for Providing Multitasking Model Learning Services]

[0444] Hereinafter, a method for providing a multi-tasking model learning service in which a computing system (1000) according to one embodiment of the present invention automatically determines learning coefficients between multiple tasks through a data-driven technique will be described in detail.

[0445] Here, for reference, data-driven techniques are training methods in which a model automatically learns the patterns and relationships that exist within given data, and they refer to learning that utilizes a large amount of data instead of explicitly defined rules or mathematical models.

[0446] At this time, the computing system (1000) according to one embodiment of the present invention can implement multi-tasking model learning (hereinafter, data-driven training) based on the data-centric technique as described above based on data for validation of the prediction performance of the learned multi-tasking learning model (MtLM) (hereinafter, validation data) and / or the aforementioned material property relationship information.

[0447] Below, a multi-task model learning method according to a data-centric technique is described in more detail with reference to the attached drawings, and any explanation that overlaps with the above may be summarized or omitted.

[0448] In detail, a computing system (1000) according to one embodiment of the present invention can perform 1) data-driven training based on property relationship information.

[0449] FIG. 23 illustrates a flowchart for explaining a multi-tasking model learning method based on a data-centric technique based on property relationship information according to one embodiment of the present invention.

[0450] In more detail, referring to FIG. 23, a computing system (1000) according to one embodiment of the present invention can perform a step (S301) of initializing a multi-tasking learning model (MtLM), a step (S303) of acquiring experimental data, a step (S305) of training the multi-tasking learning model (MtLM) based on the acquired experimental data, and a step (S307) of providing a trained multi-tasking learning model (MtLM).

[0451] At this time, a specific method in which the computing system (1000) according to one embodiment of the present invention performs the above-described steps S301 to S307 can be implemented in the same manner by applying the description in the above-described steps S101 to S107, and elements that are different from the above-described description will be described below with emphasis.

[0452] Figure 24 illustrates a conceptual diagram for explaining a multi-tasking model learning method based on a data-centric technique based on property relationship information according to one embodiment of the present invention.

[0453] Specifically, referring to FIG. 24, in an embodiment, the computing system (1000) trains the multi-task learning model (MtLM) described above (S105, S305), and the learning coefficient ( ) can be trained.

[0454] Here, the learning rate according to the embodiment may be a hyperparameter that determines the transfer rate of knowledge data between a given source task and target task pair. In other words, the learning rate may be a value that expresses the degree of transferability of knowledge data between a specific source task and target task pair.

[0455] These learning coefficients are parameters that determine the movement size / speed when updating parameters using gradient descent during the model training process, and can affect model performance and convergence speed by adjusting the amount of parameter updates at each training stage.

[0456] Depending on the embodiment, the learning coefficients may be initialized to a predetermined value based on user input and / or a pre-built algorithm.

[0457]

[0458] *In detail, in the embodiment, the computing system (1000) can perform parameter updates according to given learning coefficients and experimental data in the step (S105, S305) of training a multi-task learning model (MtLM).

[0459] At this time, in the embodiment, the computing system (1000) can update the learning coefficient described above based on the property relationship information acquired in the step (S103, S303) of acquiring the experimental data described above.

[0460] Here, in other words, the property relationship information according to the embodiment is information that specifies the relationship between different properties, and may include, as a specific example, information on properties related to a specific property, information on the properties of the relationship (e.g., conflict, similarity, correlation, cause and effect, independence, proportionality, and / or inverse relationship), and / or information on the degree of association (association strength) according to the properties of the determined relationship.

[0461] In more detail, in an embodiment, the computing system (1000) can automatically determine and update learning coefficients based on the relationship attribute information and / or the degree of association (intensity of association) information of the material relationship information.

[0462] As an example, the computing system (1000) can determine and update a learning coefficient in proportion to the degree of correlation information of the material property relationship information.

[0463] For example, the computing system (1000) can perform training to update parameters so that the learning coefficient value between a first property (e.g., boiling point) and a second property (e.g., melting point) with a high degree of correlation increases, and the learning coefficient value between a first property (e.g., boiling point) and a third property (e.g., refractive index) with a low degree of correlation decreases.

[0464] Thus, in the embodiment, the computing system (1000) can perform learning, that is, data-driven training based on property relationship information, that determines and updates learning coefficients between a given source task and target task pair according to property relationship information (particularly, degree of association information).

[0465] In this way, the computing system (1000) according to one embodiment of the present invention can implement learning that automatically determines the degree of transfer of knowledge data between the source task and the target task by reflecting relationship information between the source task and the target task.

[0466] Therefore, the computing system (1000) can effectively replace the method of manually setting hyperparameters, thereby increasing the accuracy of learning coefficient values, and at the same time, can implement fast and easy prediction for learning coefficients that increase exponentially when performing multi-task learning based on a large number of tasks.

[0467] Meanwhile, a computing system (1000) according to one embodiment of the present invention can perform 2) data-driven training based on verification data.

[0468] FIG. 25 illustrates a flowchart for explaining a multitasking model learning method according to a data-centric technique based on verification data according to an embodiment of the present invention, FIG. 26 illustrates a first conceptual diagram for explaining a multitasking model learning method according to a data-centric technique based on verification data according to an embodiment of the present invention, and FIG. 27 illustrates a second conceptual diagram for explaining a multitasking model learning method according to a data-centric technique based on verification data according to an embodiment of the present invention.

[0469] In detail, referring to FIGS. 25 to 27, a computing system (1000) according to an embodiment of the present invention may perform a step (S401) of performing a first optimization for a multi-tasking learning model (MtLM), a step (S403) of acquiring predetermined verification data, a step (S405) of performing a second optimization for the multi-tasking learning model (MtLM) based on the acquired verification data, and a step (S407) of providing a second optimized multi-tasking learning model (MtLM).

[0470] In other words, the following will focus on elements that are different from the previously described explanation, and explanations that overlap with the previously described contents may be summarized or omitted.

[0471] In detail, in an embodiment, the computing system (1000) can perform a first optimization for a multi-task learning model (MtLM). (S401)

[0472] In more detail, in the embodiment, the computing system (1000) can train a multi-tasking learning model (MtLM) according to the multi-tasking model pre-training method according to steps S101 to S107 described above.

[0473] In this process, the computing system (1000) can calculate an integrated loss according to an embodiment of the present invention based on given experimental data, and perform parameter update based on the calculated integrated loss and given learning coefficients (e.g., initialized learning coefficients, etc.).

[0474] Additionally, in the embodiment, the computing system (1000) can obtain predetermined verification data. (S403)

[0475] Here, validation data can refer to data used to validate the predictive performance of a pre-trained multi-task learning model (MtLM). In other words, validation data can be data used to validate the predictive performance of the first optimized multi-task learning model (MtLM).

[0476] In an embodiment, such verification data may include data in the same format as the experimental data described above.

[0477] In detail, in an embodiment, the computing system (1000) can obtain verification data as described above based on a predetermined user input and / or connection with an external server.

[0478] Additionally, in the embodiment, the computing system (1000) can perform secondary optimization for the multi-task learning model (MtLM) based on the acquired verification data. (S405)

[0479] In detail, in the embodiment, the computing system (1000) learns the learning coefficients of the first optimized multi-task learning model (MtLM) based on the verification data acquired as above. ) can be used to perform training to update the training data.

[0480] In other words, the computing system (1000) can perform training to update learning coefficients between a given pair of source tasks and target tasks within the first optimized multi-task learning model (MtLM) based on the acquired verification data.

[0481] In more detail, in an embodiment, the computing system (1000) can calculate a validation loss based on validation data.

[0482] Here, the validation loss according to the embodiment may be an indicator for evaluating how well the model generalizes to new data (in the embodiment, validation data). In the embodiment, the computing system (1000) may determine that the model is not overfitting and is performing optimally as the validation loss decreases.

[0483] Specifically, in an embodiment, the computing system (1000) can perform predictions based on verification data using the parameters updated in the first optimization process described above.

[0484] Additionally, in the embodiment, the computing system (1000) predicts a predicted value based on the verification data ( ) and actual value( , the validation loss can be calculated by calculating the difference between the label values.

[0485] At this time, the computing system (1000) can calculate the verification loss described above according to [Mathematical Formula 8] below.

[0486] [Equation 8]

[0487]

[0488] Here, 'N' in [Mathematical Formula 8] represents the number of data points, and 'MSE' represents the mean squared error.

[0489] Additionally, in the embodiment, the computing system (1000) can update the learning coefficient (i.e., the transition rate) based on the calculated verification loss.

[0490] As an example, the computing system (1000) can calculate a gradient based on a gradient descent method according to a verification loss and update a learning coefficient based on the calculated gradient.

[0491] At this time, in the embodiment, the computing system (1000) can perform a learning coefficient update based on verification loss according to [Mathematical Formula 9] below.

[0492] [Equation 9]

[0493]

[0494] Here, ' in [Equation 9] ' is the learning rate, ' ' means the gradient of the learning coefficients with respect to the verification loss.

[0495] At this time, in the embodiment, the computing system (1000) can perform an update to adjust the learning coefficients in a direction to minimize the verification loss.

[0496] Thus, the computing system (1000) can perform data-driven training based on verification data, that is, learning that determines and updates learning coefficients between a given source task and target task pair according to verification data.

[0497] Additionally, in the embodiment, the computing system (1000) can repeatedly perform the first optimization and the second optimization described above p (p>=2) times through a predetermined loop algorithm.

[0498] Accordingly, the computing system (1000) can alternately optimize and update the parameters and learning coefficients of the multi-task learning model (MtLM) each time verification is repeated.

[0499] In this way, the computing system (1000) according to one embodiment of the present invention can perform a first optimization for updating model parameters through a pre-learning process for a multi-task learning model (MtLM) in order to train predetermined inter-task learning coefficients according to a data-centric technique during multi-task learning through geometric alignment of an integrated latent space (M), and can perform a second optimization for updating predetermined inter-task learning coefficients through a verification process for the multi-task learning model (MtLM).

[0500] In this way, the computing system (1000) implements model learning that updates learning coefficients according to the above two-level optimization, thereby effectively replacing the hyperparameter setting method that relies on manual design, and can derive optimal learning coefficients (i.e., transfer ratios) more accurately and efficiently.

[0501] In addition, through this, the computing system (1000) can implement faster and easier prediction for learning coefficients that increase exponentially when performing multi-tasking learning based on a large number of tasks.

[0502] In addition, the computing system (1000) can minimize the gap between the learning coefficient predicted based on the tasks and the actual data distribution when the given source task and target task pair do not represent the data population corresponding to the tasks.

[0503] Thus, the computing system (1000) can directly support the improvement of learning convergence speed and performance of the model based on the optimal learning coefficient derived as above.

[0504] In the above, the embodiments have been described separately as described above for effective explanation, but various embodiments may be possible, such as at least some of the embodiments may be organically combined and operated with each other.

[0505] Additionally, in an embodiment, the computing system (1000) can provide a second optimized multi-task learning model (MtLM). (S407)

[0506] In detail, in an embodiment, the computing system (1000) can provide the second optimized multi-task learning model (MtLM) according to a predetermined method as described above.

[0507] In an embodiment, the computing system (1000) may provide a multi-task learning model (MtLM) trained according to an embodiment of the present invention (i.e., a first-order optimization and a second-order optimization multi-task learning model (MtLM)) in conjunction with a predetermined application service (e.g., a material synthesis / evaluation service, a material property prediction service, and / or an optimal material recommendation service, etc.).

[0508] As a specific example, the computing system (1000) may provide a multi-task learning model (MtLM) trained through a service that predicts multiple properties for a specific material, or may provide a multi-task learning model (MtLM) trained through a service that predicts a specific material for multiple properties.

[0509] In this way, the computing system (1000) according to one embodiment of the present invention performs multi-task model learning that automatically determines learning coefficients (i.e., transfer ratios) between multiple tasks through a data-driven technique (in an embodiment, material property relationship information and / or verification data, etc.), thereby efficiently obtaining optimal learning coefficients even in a large-scale task space, and as a result, improving model scalability while accelerating training convergence and improving performance in transfer learning for multi-task learning.

[0510] Above, the method and system for learning a multi-tasking model using a data-centric technique according to one embodiment of the present invention can effectively replace the hyperparameter setting method that relies on manual design by automatically determining learning coefficients between multiple tasks using a data-centric technique, thereby having the effect of deriving optimal learning coefficients (i.e., transfer ratios) more accurately and efficiently.

[0511] Therefore, the multi-task model learning method and system using a data-centric technique according to one embodiment of the present invention have the effect of implementing training convergence acceleration and performance improvement in transfer learning for multi-task learning.

[0512] In addition, the method and system for learning a multi-task model using a data-centric technique according to one embodiment of the present invention automatically performs a predetermined data-centric learning coefficient update according to a two-stage optimization including a first optimization performed in a pre-learning process and a second optimization performed in a model verification process, thereby supporting automated updates for learning coefficients that increase exponentially in a large task space, thereby improving model scalability.

[0513] In addition, the method and system for learning a multi-tasking model using a data-centric technique according to one embodiment of the present invention have the effect of minimizing the difference between the learning coefficient predicted based on the tasks and the actual data distribution when the given source task and target task pair do not represent the data population corresponding to the tasks.

[0514]

[0515] Meanwhile, the embodiments of the present invention described above may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. Hardware devices may be changed into one or more software modules to perform processing according to the present invention, and vice versa.

[0516] The specific implementations described in the present invention are exemplary embodiments and do not limit the scope of the present invention in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the lines connecting or connecting members between components illustrated in the drawings are merely representative of functional connections and / or physical or circuit connections, and may be replaced or represented as various additional functional connections, physical connections, or circuit connections in an actual device. In addition, unless specifically mentioned as “essential,” “important,” etc., a component may not be absolutely necessary for the application of the present invention.

[0517] Although the detailed description of the present invention has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art or having ordinary knowledge in the art that various modifications and changes can be made to the present invention without departing from the spirit and technical scope of the present invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

[0518] The present invention relates to a multi-tasking model learning method and system using a data-centric technique, and is applicable to the artificial intelligence industry, and thus has industrial applicability.

Claims

1. A method for performing multi-tasking model learning according to data-centric techniques in a computing system including memory and processor, A step of performing a first optimization to update the parameters of a multi-tasking model based on predetermined experimental data; A step of obtaining validation data, which is data for validating the prediction performance of the first optimized multi-tasking model; A step of performing a second optimization for updating the learning rate of the first optimized multi-task model based on the acquired verification data; and comprising a step of providing the second optimized multitasking model; A multi-task model learning method using data-driven techniques.

2. In paragraph 1, The step of performing the above first optimization is: A step of obtaining a geometric alignment vector, which is a vector that supports geometric alignment in the integrated latent space based on the above experimental data, A step of calculating geometric alignment loss based on the geometric alignment vector obtained above, A step of updating the parameters of the multi-tasking model based on the geometric alignment loss produced above. A multi-task model learning method using data-driven techniques.

3. In paragraph 1, The above learning coefficient is, In the first optimization process above, when performing transfer learning, a hyperparameter that determines the transfer ratio between the pair of the source task, which is the task that transfers data, and the target task, which is the task that receives data, is A multi-task model learning method using data-driven techniques.

4. In paragraph 3, The step of performing the above second optimization is: A step of calculating validation loss, which is an indicator for evaluating the generalization performance of the multi-task model, based on the updated parameters and the validation data based on the first optimization, including a step of updating the learning coefficient based on the above-described verification loss. A multi-task model learning method using data-driven techniques.

5. In paragraph 4, The step of updating the above learning coefficient is: including a step of adjusting the learning coefficient in a direction to minimize the verification loss. A multi-task model learning method using data-driven techniques.

6. In paragraph 1, Further comprising a step of repeatedly performing the first optimization and the second optimization based on a predetermined loop algorithm. A multi-task model learning method using data-driven techniques.

7. In paragraph 1, The above experimental data and verification data, It includes material-specific characteristic information, which is information that specifies the unique characteristics of a given material, and material property-specific information, which is information that specifies the characteristic values ​​that the material has for multiple physical properties. A multi-task model learning method using data-driven techniques.

8. In paragraph 7, A method further comprising: obtaining property relationship information, which is information including data specifying properties associated with a given property, data specifying properties of a relationship between the associated properties, and data specifying the degree of association according to the properties of the relationship. A multi-task model learning method using data-driven techniques.

9. In paragraph 8, Further comprising a step of updating the learning coefficient based on the above property relationship information. A multi-task model learning method using data-driven techniques.

10. At least one memory; and At least one processor configured to read at least one application stored in the memory and perform multi-tasking model learning according to a data-centric technique; The instructions of the above processor are: A step of performing a first optimization to update the parameters of a multi-tasking model based on a predetermined experimental data, A step of obtaining validation data, which is data for validating the prediction performance of the first optimized multi-tasking model, A step of performing a second optimization to update the learning rate of the first optimized multi-tasking model based on the verification data obtained above, A command comprising a step of providing the second optimized multitasking model. A multi-task model learning system based on data-centric techniques.

Images