The Method, System, and Computer-readable Storage Medium Of Automatic Generating Of Learning Inference Data Using Connectors, And Self-Learning Of LLM Using The Same

Abstract

The present invention relates to a method for automatically generating inference data for learning using a connector, a self-learning method for an LLM using the same, a system, and a computer-readable storage medium. More specifically, the invention relates to a method for automatically generating inference data for learning using a connector, a self-learning method for an LLM using the same, a system, and a computer-readable storage medium, wherein a task question extracted from a task dataset is input into an LLM to derive a first answer, and if the first answer is not the correct answer, the corresponding task question, the first answer, and a connector are input into the LLM to derive a second answer, and if the second answer is the correct answer, inference data for learning is derived.

Description

Technology Field

[0001] The present invention relates to a method for automatically generating inference data for learning using a connector, a self-learning method for an LLM using the same, a system, and a computer-readable storage medium. More specifically, the invention relates to a method for automatically generating inference data for learning using a connector, a self-learning method for an LLM using the same, a system, and a computer-readable storage medium, wherein a task question extracted from a task dataset is input into an LLM to derive a first answer, and if the first answer is not the correct answer, the corresponding task question, the first answer, and a connector are input into the LLM to derive a second answer, and if the second answer is the correct answer, inference data for learning is derived. Background Technology

[0003] Recently, test-time scaling methods have been gaining attention due to the difficulty of scaling during training. Notably, OpenAI's o1 model demonstrated high performance using this method, but since its specific implementation was not disclosed, many studies have attempted similar approaches, particularly utilizing reinforcement learning (RL) and Monte Carlo tree search (MCTS). However, these methods have shown limitations due to the cost associated with the inference process.

[0004] At this time, the study S1: Simple Test-time Scaling published by Stanford demonstrated excellent performance by fine-tuning a 32B model using only 1,000 high-quality inference datasets, and LIMO: Less is More for Reasoning solved the cold-start problem of the DeepSeek-R1 model using 817 data points and generated high-quality inference data through the curation of 1,000 datasets. Through these studies, it can be seen that if appropriate inference data is available, a model with excellent performance can be trained.

[0005] While traditional inference datasets focused on simply deriving the correct answer during the problem-solving process, high-quality inference datasets go beyond simple answer generation to provide a much longer and more detailed thought process that resembles real human thinking. This includes additional thought processes such as self-reflection, verification, and aha points. In other words, these are datasets that require more thought and time to reach the correct answer, and are referred to as Long CoT datasets.

[0006] However, it is very difficult for humans to directly generate such high-quality inference data, and reviewing the quality of the generated data requires significant effort. Under these circumstances, there is a need for technology that generates diverse inference data using connectors without requiring extensive resources, and enables the LLM to self-learn and progressively improve through this inference data. The problem to be solved

[0008] The present invention aims to provide a method for automatically generating inference data for learning using a connector, a self-learning method for an LLM using the same, a system, and a computer-readable storage medium. More specifically, the invention provides a method for automatically generating inference data for learning using a connector, a self-learning method for an LLM using the same, a system, and a computer-readable storage medium. Specifically, the invention provides a method for automatically generating inference data for learning using a connector, a self-learning method for an LLM using the same, a system, and a computer-readable storage medium. The method involves inputting a task question extracted from a task dataset into an LLM to derive a first answer, inputting the corresponding task question, the first answer, and a connector into the LLM to derive a second answer when the first answer is not the correct answer, and deriving inference data for learning when the second answer is the correct answer. means of solving the problem

[0010] To solve the above-mentioned problem, in one embodiment of the present invention, a method for automatically generating training inference data using a connector performed in a server system comprising one or more processors and one or more memories comprises: a task question extraction step of extracting a plurality of task questions from a task dataset including a task question and a task answer corresponding to the task question; a first answer derivation step of inputting the extracted task questions into an LLM outside or inside the server system to derive a first answer; a first answer verification step of comparing the first answer with the task answer corresponding to the extracted task question to determine whether the first answer is a correct answer; and, if the first answer is not a correct answer, a second answer derivation step of inputting a first connector randomly determined among the task question, the first answer, and a preset plurality of connectors into the LLM to derive a second answer. A method for automatically generating inference data for training is provided, comprising: a second answer verification step for comparing the second answer with the task answer to determine whether the second answer is the correct answer.

[0011] In one embodiment of the present invention, the method for automatically generating the learning inference data may further include: a third answer derivation step in which, when the second answer is not the correct answer, a second connecting word randomly determined among the task question, the first answer, the first connecting word, the second answer, and a plurality of pre-set connecting words is input into the LLM to derive a third answer; and a third answer verification step in which the third answer is compared with the task correct answer to determine whether the third answer is the correct answer.

[0012] In one embodiment of the present invention, the second answer verification step, when the second answer is correct, derives the task question, the first answer, the first connecting word, and the second answer as a set to derive the inference data for learning, and the method for automatically generating the inference data for learning may further include a learning step of learning the LLM using the inference data for learning.

[0013] In one embodiment of the present invention, the second answer verification step, when the second answer is correct, derives the task question, the first answer, the first connecting word, and the second answer as a set to derive the inference data for training, and the method for automatically generating the inference data for training may further include a training step of training a target model other than the LLM using the inference data for training.

[0014] In one embodiment of the present invention, the method for automatically generating the learning inference data comprises inputting each of a plurality of task questions included in the task dataset into an LLM to detect task questions for which a non-correct answer is derived, applying each of a plurality of pre-set connectors to each of the detected task questions to derive a modified answer, and based on the modified answer, the plurality of connectors may be classified into connectors for which a correct answer is derived and connectors for which a non-correct answer is derived.

[0015] In one embodiment of the present invention, the method for automatically generating the learning inference data comprises, for a plurality of preset connectors, generating a plurality of branch nodes by applying each connector to a non-correct answer among the answers derived from the LLM, and forming a tree by repeatedly applying each connector to a branch node containing a non-correct answer among the plurality of branch nodes to generate a plurality of branch nodes, wherein the plurality of branch nodes can be classified into a node containing a correct answer and a node containing a non-correct answer.

[0016] In one embodiment of the present invention, the method for automatically generating the training inference data can generate branch nodes until the tree reaches a preset depth, detect a leaf node containing a correct answer among the leaf nodes of the tree, and derive the training inference data by configuring the path from the detected leaf node to the root node into a single set.

[0017] In order to solve the above problems, one embodiment of the present invention provides a server system that includes one or more processors and one or more memories and performs a method for automatically generating inference data for learning using a connector, comprising: a task question extraction unit that extracts a plurality of task questions from a task dataset including a task question and a task answer corresponding to the task question; a first answer derivation unit that inputs the extracted task questions into an LLM outside or inside the server system to derive a first answer; a first answer verification unit that compares the first answer with the task answer corresponding to the extracted task question to determine whether the first answer is a correct answer; a second answer derivation unit that inputs the task question, the first answer, and a first connector randomly determined among a plurality of preset connectors into the LLM to derive a second answer when the first answer is not a correct answer; and a second answer verification unit that compares the second answer with the task answer to determine whether the second answer is a correct answer.

[0018] In order to solve the above problem, in one embodiment of the present invention, a computer-readable storage medium for implementing a method of automatically generating learning inference data using a connector executed in a server system comprising one or more processors and one or more memories is provided, wherein the computer-readable storage medium comprises computer-executable instructions that cause the server system to perform the following steps, and the following steps include: a task question extraction step of extracting a plurality of task questions from a task dataset including a task question and a task answer corresponding to the task question; a first answer derivation step of inputting the extracted task question into an LLM outside or inside the server system to derive a first answer; and a first answer verification step of comparing the first answer with the extracted task answer corresponding to the task question to determine whether the first answer is a correct answer. A computer-readable storage medium is provided, comprising: a second answer derivation step in which, when the first answer is not the correct answer, a first connector randomly determined among the task question, the first answer, and a plurality of preset connectors is input into the LLM to derive a second answer; and a second answer verification step in which the second answer is compared with the task correct answer to determine whether the second answer is the correct answer. Effects of the invention

[0021] According to one embodiment of the present invention, the effect of automatically generating training inference data using a task dataset and a connecting word can be achieved.

[0022] According to one embodiment of the present invention, the effect of self-learning LLM through learning inference data can be achieved.

[0023] According to one embodiment of the present invention, when the LLM derives an answer that is not the correct answer to an input task question, it can apply a connecting word to derive a modified answer, thereby achieving the effect of deriving a modified answer.

[0024] According to one embodiment of the present invention, when the LLM derives a correct answer to an input task question, it can produce the effect of deriving inference data for training.

[0025] According to one embodiment of the present invention, the effect of being able to train an LLM and a target model using training inference data can be achieved.

[0026] According to one embodiment of the present invention, a branching node can be generated by applying a connecting word to a non-correct answer among the answers derived from LLM.

[0027] According to one embodiment of the present invention, a tree including branch nodes is generated based on a plurality of connecting words, and the effect of deriving inference data for training based on the tree can be achieved.

[0028] According to one embodiment of the present invention, the effect of automatically generating inference data that satisfies diversity and quality using connecting words can be achieved.

[0029] According to one embodiment of the present invention, the effect of continuously developing a model with excellent inference performance through self-learning of the LLM can be achieved. Brief explanation of the drawing

[0031] FIG. 1 schematically illustrates the internal configuration of a server system according to one embodiment of the present invention. FIG. 2 schematically illustrates the steps of a method for automatically generating learning inference data according to an embodiment of the present invention. FIG. 3 schematically illustrates the process of performing a task question extraction step according to an embodiment of the present invention. FIG. 4 schematically illustrates the process of performing the first answer derivation step according to one embodiment of the present invention. FIG. 5 schematically illustrates the process of performing the second answer derivation step and the second answer verification step according to one embodiment of the present invention. FIG. 6 schematically illustrates inference data for learning according to one embodiment of the present invention. FIG. 7 schematically illustrates the process of performing the step of creating a branch node according to one embodiment of the present invention. FIG. 8 schematically illustrates the process of performing a learning step according to one embodiment of the present invention. FIG. 9 illustrates, in an exemplary manner, the internal configuration of a computing device according to one embodiment of the present invention. Specific details for implementing the invention

[0032] Hereinafter, various embodiments and / or aspects are disclosed with reference to the drawings. For illustrative purposes, numerous specific details are disclosed in the following description to aid in a general understanding of one or more aspects. However, it will also be recognized by those skilled in the art that these aspects may be practiced without such specific details. The following description and the accompanying drawings describe specific exemplary aspects of one or more aspects in detail. However, these aspects are exemplary, and some of the various methods in the principles of the various aspects may be used, and the description is intended to include all such aspects and their equivalents.

[0033] Additionally, terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0034] Furthermore, in the embodiments of the present invention, all terms used herein, including technical or scientific terms, unless otherwise defined, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in the embodiments of the present invention.

[0035] The "user terminal" mentioned below may be implemented as a computer or portable terminal capable of connecting to a server or other terminal via a network. Here, the computer includes, for example, a notebook, desktop, or laptop equipped with a web browser, and the portable terminal may include, for example, all types of handheld-based wireless communication devices that ensure portability and mobility, such as smartphones, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet), and BLE Beacon (Bluetooth Low Energy Beacon) terminals. In addition, the “network” can be implemented as a wired network such as a Local Area Network (LAN), Wide Area Network (WAN), or Value Added Network (VAN), or as any type of wireless network such as a mobile radio communication network or a satellite communication network.

[0037] FIG. 1 schematically illustrates the internal configuration of a server system (1000) according to one embodiment of the present invention.

[0039] As illustrated in FIG. 1, the server system (1000) comprises: a task question extraction unit (100) that performs a task question extraction step of extracting a plurality of task questions from a task dataset including a task question and a task answer corresponding to the task question; a first answer derivation unit (200) that performs a first answer derivation step of inputting the extracted task question into an LLM (2000) outside or inside the server system (1000) to derive a first answer; and a first answer verification unit (300) that performs a first answer verification step of comparing the first answer with the extracted task answer corresponding to the task question to determine whether the first answer is a correct answer. If the above first answer is not the correct answer, a second answer derivation unit (400) performs a second answer derivation step in which a first connector randomly determined among the above task question, the above first answer, and a plurality of preset connectors is input into the LLM (2000) to derive a second answer; a second answer verification unit (500) performs a second answer verification step in which the second answer is compared with the above task correct answer to determine whether the second answer is the correct answer; and a learning unit (600) performs a learning step in which the LLM (2000) is trained using the above learning inference data, or a target model (3000) other than the LLM (2000) is trained using the above learning inference data.

[0041] Specifically, each component included in the server system (1000) illustrated in FIG. 1 performs the role of controlling the operation of the server system (1000) that performs the method of automatically generating learning inference data of the present invention.

[0042] More specifically, the task question extraction unit (100) of the server system (1000) can extract a plurality of task questions from a task dataset including task questions and task answers corresponding to the task questions.

[0043] In one embodiment of the present invention, the task question extraction step may extract all task questions included in the task dataset, or may randomly extract a plurality of task questions from all task questions included in the task dataset.

[0045] The first answer derivation unit (200) of the above server system (1000) can derive a first answer by inputting the extracted task question into an LLM (2000) outside or inside the server system (1000).

[0046] In one embodiment of the present invention, the LLM (2000) may be located outside or inside the server system (1000) and may correspond to a model that derives a first answer including an answer to the task question when the task question is input. The first answer may include all answers derived from the LLM (2000) in the first answer derivation step.

[0048] The first answer verification unit (300) of the above server system (1000) can determine whether the first answer is correct by comparing the first answer with the task answer corresponding to the extracted task question.

[0049] In one embodiment of the present invention, when the first answer is the correct answer, the task question and the first answer can be formed as a set to derive inference data for learning.

[0051] The second answer derivation unit (400) of the server system (1000) can derive a second answer by inputting the task question, the first answer, and a first connecting word randomly determined among a plurality of pre-set connecting words into the LLM (2000) when the first answer is not the correct answer.

[0052] In one embodiment of the present invention, the LLM (2000) is located outside or inside the server system (1000) and may correspond to a model that derives a second answer including an answer to the task question when the task question, the first answer, and the first connector are input. The second answer may include all answers derived from the LLM (2000) in the second answer derivation step. In this case, the LLM deriving the first answer and the LLM deriving the second answer correspond to the same model.

[0054] The second answer verification unit (500) of the above server system (1000) can compare the second answer with the task answer to determine whether the second answer is the correct answer.

[0055] In one embodiment of the present invention, the server system (1000) may further perform: a third answer derivation step in which, if the second answer is not the correct answer, inputs the task question, the first answer, the first connecting word, the second answer, and a second connecting word randomly determined among a plurality of pre-set connecting words into the LLM (2000) to derive a third answer; and a third answer verification step in which the third answer is compared with the task correct answer to determine whether the third answer is the correct answer.

[0056] In this case, if the above second answer is the correct answer, the above task question, the above first answer, the above first connecting word, and the above second answer can be formed as a set to derive inference data for training.

[0058] The learning unit (600) of the above server system (1000) can train the LLM (2000) using the above training inference data, or train a target model (3000) other than the LLM (2000) using the above training inference data.

[0059] In one embodiment of the present invention, the LLM (2000) corresponds to the LLM that derived the first answer and the second answer, and the target model (3000) corresponds to a model that is located outside or inside the server system (1000) and is not the LLM (2000), and is intended to be trained using the learning inference data, and the target model (3000) may include the LLM.

[0061] Preferably, the method for automatically generating learning inference data of the present invention inputs each of a plurality of task questions included in the task dataset into an LLM outside or inside a server system to detect task questions from which a non-correct answer is derived, applies each of a plurality of pre-set connectors to each of the detected task questions to derive a modified answer, and based on the modified answer, the plurality of connectors can be classified into connectors that derive a correct answer and connectors that derive a non-correct answer.

[0063] In one embodiment of the present invention, the method for automatically generating the learning inference data comprises, for a plurality of preset connectors, generating a plurality of branch nodes by applying each connector to a non-correct answer among the answers derived from the LLM, and forming a tree by repeatedly applying each connector to a branch node containing a non-correct answer among the plurality of branch nodes to generate a plurality of branch nodes, wherein the plurality of branch nodes can be classified into a node containing a correct answer and a node containing a non-correct answer.

[0064] In addition, the method for automatically generating the above-mentioned training inference data can generate branch nodes until the tree reaches a preset depth, detect leaf nodes containing the correct answer among the leaf nodes of the tree, and derive training inference data by configuring the path from the detected leaf node to the root node into a single set.

[0066] FIG. 2 schematically illustrates the steps of a method for automatically generating learning inference data according to an embodiment of the present invention.

[0068] As illustrated in FIG. 2, a plurality of task questions are extracted (S100) from a task dataset containing task questions and task answers corresponding to the task questions, and one of the extracted task questions is input into an LLM (2000) outside or inside the server system (1000) to derive a first answer (S200).

[0070] At this time, the first answer can be compared with the task answer corresponding to the task question input into the LLM (2000) to determine whether the first answer is correct (S300), and if the first answer is correct, the task question and the first answer can be formed into a set to derive inference data for learning (S400).

[0071] If the above first answer is not the correct answer, a second answer can be derived (S500) by inputting the corresponding task question, the above first answer, and one first connecting word randomly determined from a plurality of preset connecting words into the LLM (2000).

[0073] At this time, the second answer can be compared with the task answer to determine whether the second answer is correct (S600), and if the second answer is correct, the task question, the first answer, the first connecting word, and the second answer can be formed into a set to derive inference data for learning (S700).

[0074] If the above second answer is not the correct answer, a third answer can be derived (S800) by inputting one second connecting word, randomly determined from the task question, the above first answer, the above first connecting word, the above second answer, and a plurality of preset connecting words excluding the above first connecting word, into the LLM (2000).

[0076] At this time, the third answer can be compared with the task answer to determine whether the third answer is the correct answer, and if the third answer is the correct answer, the task question, the first answer, the first connector, the second answer, the second connector, and the third answer can be formed as a set to derive inference data for training.

[0077] If the above third answer is not the correct answer, a fourth answer can be derived again by applying a third connector randomly determined from among the pre-set multiple connectors, excluding the above first connector and the above second connector.

[0079] Therefore, if the answer derived from the LLM (2000) based on the task answer is correct, training inference data can be derived, and if it is not correct, the connecting word can be repeatedly applied so that the process as in FIG. 2 can be repeated until the LLM (2000) derives the correct answer.

[0081] FIG. 3 schematically illustrates the process of performing a task question extraction step according to an embodiment of the present invention.

[0083] Generally, the task question extraction step can extract multiple task questions from a task dataset containing task questions and task answers corresponding to the task questions.

[0085] Specifically, a task dataset including task questions and task answers is stored in a server system (1000), and multiple task questions can be extracted from the task dataset to generate inference data for training.

[0086] In one embodiment of the present invention, the task dataset may include questions used to evaluate problem-solving abilities or comprehension, such as mathematics or logic, as in AIME, AGIEval, and OmniMath, and evaluation indicators in the form of corresponding answers. Additionally, the task questions and task answers stored in the task dataset may be periodically updated, such as by adding, modifying, and deleting.

[0088] As illustrated in FIG. 3, task questions #1 to #N and task answers #1 to #N corresponding to each task question are stored in the task dataset of the server system (1000), and a plurality of task questions including task questions #1 to #M can be extracted.

[0089] At this time, the above task answer includes an answer that is the correct answer to the corresponding task question, and N and M each correspond to a natural number greater than or equal to 1, and N and M may correspond to the same natural number or different natural numbers. Therefore, N may correspond to a natural number greater than M or equal to M.

[0090] For example, the correct answer to the task question "1+1=?" can be "2", and any answer other than the above correct answer can be considered incorrect.

[0092] FIG. 4 schematically illustrates the process of performing the first answer derivation step according to one embodiment of the present invention.

[0094] In summary, the first answer derivation step can derive a first answer by inputting the extracted task question into an LLM (2000) outside or inside the server system (1000).

[0096] Specifically, each of the extracted multiple task questions can be input into the LLM (2000) to derive a first answer for each of the task questions, and any one of the extracted multiple task questions can be input into the LLM (2000) to derive a first answer for the corresponding task question.

[0098] As illustrated in FIG. 4, task questions #1 to #M extracted in the task question extraction step can be input into an LLM (2000) outside or inside the server system (1000) to derive a first answer for each of task questions #1 to #M. At this time, the first answer in FIG. 4 may include first answer #1 to first answer #M.

[0099] In one embodiment of the present invention, the first answer may include all responses of the LLM (2000) to the input task question.

[0101] Subsequently, the derived first answer can be compared with the task answer corresponding to the task question to determine whether the first answer is the correct answer. If the first answer #1 for task question #1 is derived through LLM (2000), the first answer #1 can be compared with the task answer #1 for task question #1 stored in the task dataset to determine whether the first answer #1 is the correct answer.

[0102] In one embodiment of the present invention, a server system (1000) can determine whether the first answer #1 is the correct answer by comparing the task answer #1 and the first answer #1, or the server system (1000) can determine whether the first answer #1 is the correct answer by inputting the task answer #1 and the first answer #1 into an LLM outside or inside the server system (1000).

[0104] FIG. 5 schematically illustrates the process of performing the second answer derivation step and the second answer verification step according to one embodiment of the present invention.

[0106] In summary, FIG. 5(a) illustrates the process of performing the second answer derivation step, and FIG. 5(b) illustrates the process of performing the second answer verification step.

[0108] Specifically, the second answer derivation step can derive a second answer by inputting the task question, the first answer, and a first connecting word randomly determined among a plurality of pre-set connecting words into the LLM (2000) when the first answer is not the correct answer, and the second answer verification step can determine whether the second answer is the correct answer by comparing the second answer with the task correct answer.

[0110] At this time, if the second answer is not the correct answer, a third answer derivation step in which a second connecting word randomly determined among the task question, the first answer, the first connecting word, the second answer, and a plurality of pre-set connecting words is input into the LLM (2000) to derive a third answer; and a third answer verification step in which the third answer is compared with the task correct answer to determine whether the third answer is the correct answer may be performed in the server system (1000).

[0111] In addition, if the above second answer is the correct answer, the above task question, the above first answer, the above first connecting word, and the above second answer can be configured as a set to derive inference data for training.

[0113] As illustrated in FIG. 5(a), if the first answer is not the correct answer, the task question, the first answer, and a first connecting word randomly determined from a plurality of preset connecting words can be input into the LLM (2000) to derive a second answer. At this time, the LLM used in the second answer derivation step may correspond to the same model as the LLM used in the first answer derivation step.

[0114] In one embodiment of the present invention, the second answer may include all responses of the LLM (2000) to the corresponding task question that are output based on the input task question, the first answer, and the first connector, and the pre-set plurality of connectors may include Oh, Wait, Wait a second, But hold on, Actually, and On second thought, etc.

[0116] In addition, the aforementioned pre-set multiple connecting words may include phrases that induce the LLM to derive an inference process, and in this case, the inference process may include various inference processes such as sequential expansion, re-evaluation, double-checking, hypothesis conversion, and contradiction recognition.

[0117] For example, the connecting words in the present invention may include Let's break this down one step at a time, which induces sequential development from LLM; Hold on? let's re-evaluate our approach, which induces re-evaluation; Before moving forward, let's verify this step, which induces double-checking; Working backwards, we see that we need to, which induces hypothesis conversion; and Given this contradiction, we should revise our plan, which induces recognition of a contradiction.

[0119] Afterward, the derived second answer can be compared with the task answer corresponding to the task question to determine whether the second answer is correct. If, after the first answer #1 for task question #1 is derived through the LLM (2000), the first answer #1 is not correct, the task question #1, the first answer #1, and a randomly determined first connector can be input into the LLM (2000). When the second answer #1 is derived through the LLM (2000), the task answer #1 for task question #1 stored in the task dataset and the second answer #1 can be compared to determine whether the second answer #1 is correct.

[0120] In one embodiment of the present invention, a server system (1000) can determine whether the second answer #1 is the correct answer by comparing the task answer #1 and the second answer #1, or the server system (1000) can determine whether the second answer #1 is the correct answer by inputting the task answer #1 and the second answer #1 into an LLM outside or inside the server system (1000).

[0121] At this time, the first connecting word may correspond to any one randomly determined among a plurality of connecting words pre-set in the server system (1000), and the plurality of connecting words pre-set in the server system (1000) may be updated.

[0123] As illustrated in Fig. 5(b), if the second answer is determined to be correct through the second answer verification step, and the second answer is correct, the task question, the first answer, the first connecting word, and the second answer can be formed into a set and derived as inference data for learning.

[0124] In one embodiment of the present invention, the LLM (2000) or a target model (3000) other than the LLM (2000) can be trained using the training inference data.

[0126] If, when determining whether the second answer is correct through the second answer verification step, the second answer is not correct, the task question, the first answer, the first connector, the second answer, and a second connector randomly determined among a plurality of pre-set connectors are input into the LLM (2000) to derive a third answer, and then the third answer is compared with the task answer to determine whether the third answer is correct.

[0127] At this time, the second connector is a connector different from the first connector, and may correspond to any one of the plurality of connectors pre-set in the server system (1000) that is randomly determined excluding the first connector.

[0128] Preferably, if the answer derived from LLM (2000) is not the correct answer, a new answer can be derived by applying a connecting word, and if the new answer is also not the correct answer, the process of deriving a new answer again can be repeated by additionally applying a new connecting word. The process of deriving an answer as described above can be repeated by additionally applying a new connecting word until a correct answer is derived.

[0130] FIG. 6 schematically illustrates inference data for learning according to one embodiment of the present invention.

[0132] In summary, the method for automatically generating the above-mentioned training inference data involves inputting each of the plurality of task questions included in the task dataset into an LLM outside or inside a server system to detect task questions from which non-correct answers are derived, applying each of the plurality of pre-set connectors to each of the detected task questions to derive modified answers, and based on the modified answers, the plurality of connectors can be classified into connectors that derive correct answers and connectors that derive non-correct answers.

[0134] In one embodiment of the present invention, each of the plurality of task questions included in the task dataset is input into an LLM to detect the task question for which a non-correct answer is derived, and the corresponding task question and the corresponding answer are derived as incorrect answer samples; any one of the plurality of pre-set conjunctions is applied to the incorrect answer samples to derive a plurality of correct answers; and the correct answers are compared with the task correct answers corresponding to the task questions to determine whether the correct answers are correct.

[0135] In this case, if the above-mentioned correct answer is not the correct answer, another one of the above-mentioned multiple connectors is applied to the corresponding incorrect answer sample and the corresponding correct answer to derive an additional correct answer, and if the above-mentioned correct answer is the correct answer, the corresponding task question, the corresponding answer, the corresponding connector, and the corresponding correct answer can be configured as a set to derive inference data for training.

[0137] As illustrated in FIG. 6, training inference data can be automatically generated through the method of automatically generating training inference data of the present invention. In FIG. 6, (1) corresponds to a task question, (2) corresponds to a first answer, (3) corresponds to a first connecting word, and (4) corresponds to a second answer.

[0138] If a task question "How many r in raspberry" is entered into the LLM (2000) as in (1), a first answer as in (2) may be output. In this case, if the task answer corresponding to the task question is "3", the first answer may be judged not to be correct because the answer "2" was output in (2).

[0139] Preferably, if the first answer is not the correct answer, the corresponding task question, the first answer, and the first connecting word corresponding to (1) to (3) can all be re-entered into the corresponding LLM (2000) to derive the second answer.

[0141] Therefore, by applying the first connecting word "wait" as in (3) and inputting all of (1) to (3) into the LLM (2000), a second answer as in (4) can be output. At this time, the task answer corresponding to the task question is "3", and since the answer "3" was output in (4), the second answer can be determined to be the correct answer.

[0142] Subsequently, the task questions, first answers, first connecting words, and second answers corresponding to (1) to (4) can all be formed into a single set to derive training inference data. That is, the entirety of (1) to (4) can correspond to a single training inference data.

[0144] In one embodiment of the present invention, each of the answers including the first answer and the second answer may include inference content and a result, the inference content may include a process for deriving the answer, and the result may include an answer to the task question.

[0145] For example, regarding the task question "How many r in raspberry" in Fig. 6, a first answer "2" can be derived through a process similar to the reasoning content of (2), and "2" can correspond to the above result.

[0147] FIG. 7 schematically illustrates the process of performing the step of creating a branch node according to one embodiment of the present invention.

[0149] In summary, the method for automatically generating the above-mentioned training inference data is to form a tree by applying each of the pre-set multiple connecting words to the answers that are not correct among the answers derived from the LLM (2000) to generate multiple branch nodes, and repeatedly applying each of the connecting words to the branch nodes containing the answers that are not correct among the multiple branch nodes to generate multiple branch nodes, and the multiple branch nodes can be classified into nodes containing correct answers and nodes containing answers that are not correct.

[0150] In addition, branch nodes are generated until the tree reaches a preset depth, leaf nodes containing the correct answer among the leaf nodes of the tree are detected, and the path from the detected leaf node to the root node is configured as a set to be derived as inference data for training.

[0152] As illustrated in FIG. 7, the tree illustrated in FIG. 7 may correspond to a tree generated for a first answer that is not the correct answer, and the tree may include a plurality of branch nodes including nodes 1 through 16.

[0153] Specifically, for multiple extracted task questions, a first answer that is not the correct answer can be detected among the first answers derived from the LLM (2000). Afterwards, multiple branch nodes can be generated by applying a connecting word to each detected first answer that is not the correct answer. At this time, Node 1 may correspond to a branch node for one first answer that is not the correct answer.

[0154] If there are a total of three connectors including connector #1 to connector #3 in the server system (1000), nodes 2 through 4 can be created by applying each of the three connectors to node 1 to derive a second answer. For example, node 2 for answer #1 can be created by applying connector #1 to node 1 to derive answer #1, node 2 for answer #1 can be created, node 3 for answer #2 can be created by applying connector #2 to node 1 to derive answer #2, node 4 for answer #3 can be created by applying connector #3 to node 1 to derive answer #3.

[0155] Preferably, when each of connector #1 to connector #3 is applied to node 1, different answers may be derived, and each connector may be classified into a connector that derives a correct answer and a connector that derives a non-correct answer.

[0157] In this case, among nodes 2 through 4, if the second answer #1 corresponding to node 2 is the correct answer, the second answer #2 corresponding to node 3 is not the correct answer, and the second answer #3 corresponding to node 4 is not the correct answer, a branch node can be created by applying additional connecting words to nodes 3 and 4 corresponding to the incorrect answers to derive the third answer.

[0158] As shown in FIG. 6, nodes 5 through 7 can be generated by applying each of conjunctions #1 through #3 to node 3, and nodes 8 through 10 can be generated by applying each of conjunctions #1 through #3 to node 4. For example, node 5 can be generated for third answer #1 after applying conjunction #1 to node 3, node 6 can be generated for third answer #2 after applying conjunction #2 to node 3 to derive third answer #2, and node 7 can be generated for third answer #3 after applying conjunction #3 to node 3 to derive third answer #3.

[0159] In one embodiment of the present invention, when a second answer that is not the correct answer is derived when connecting word #1 is applied, since connecting word #1 has already been applied, connecting words #2 to #3 can each be additionally applied to derive two third answers, or connecting words #1 to #3 can each be additionally applied to derive three third answers.

[0161] In addition, among the nodes 5 through 10 generated in this way, if the third answer corresponding to each of nodes 6, 7, 8, and 10 is the correct answer, and the third answer corresponding to each of nodes 5 and 9 is not the correct answer, a branch node can be generated by applying a connecting word to nodes 5 and 9 corresponding to the answer that is not the correct answer to derive a fourth answer.

[0162] At this time, nodes 11 through 13 can be created by applying each of conjunction #1 through #3 to node 5, and nodes 14 through 16 can be created by applying each of conjunction #1 through #3 to node 9. For example, node 11 for 4th Answer #1 can be created by applying conjunction #1 to node 5 to derive 4th Answer #1, node 11 for 4th Answer #1 can be created by applying conjunction #2 to node 5 to derive 4th Answer #2, node 12 for 4th Answer #2 can be created, and node 13 for 4th Answer #3 can be created by applying conjunction #3 to node 5 to derive 4th Answer #3.

[0163] In one embodiment of the present invention, when a third answer that is not the correct answer is derived by applying conjunction #1 and additionally applying conjunction #2, one fourth answer can be derived by additionally applying conjunction #3 since conjunction #1 and conjunction #2 have already been applied, or three fourth answers can be derived by additionally applying each of conjunction #1 to conjunction #3.

[0165] Preferably, when a correct answer is derived, no further branch nodes are generated, but when an incorrect answer is derived, a connector is applied to generate additional branch nodes. Therefore, the connector can be repeatedly applied until a correct answer is derived.

[0167] Additionally, D1 to D4 in FIG. 7 may correspond to the depth of the tree or the depth of the corresponding branch node. For example, the tree illustrated in FIG. 7 corresponds to a tree with a total depth of 4, the depth of node 1 corresponds to 1, the depth of nodes 2 through 4 corresponds to 2, the depth of nodes 5 through 10 corresponds to 3, and the depth of nodes 11 through 16 corresponds to 4.

[0168] In one embodiment of the present invention, branch nodes are generated until the tree reaches a preset depth according to the settings of the server system (1000) or the user, and then leaf nodes containing the correct answer among the leaf nodes of the tree are detected, and the path from the detected leaf node to the root node is configured as a set to derive inference data for training. Accordingly, the inference data for training generated may vary depending on the settings.

[0170] For example, in the case where the preset depth is 4, branch nodes are generated until the depth reaches 4 as shown in Fig. 7, and then nodes 11, 12, 14, 15, and 16, which contain the correct answer among the leaf nodes in the tree, are detected, and the path from each detected branch node to the root node is formed into a set to derive inference data for training.

[0171] In this case, for Node 11, Node 11, Node 5, Node 3, and Node 1 can be formed into a set to derive training inference data, and for Node 16, Node 16, Node 9, Node 4, and Node 1 can be formed into a set to derive training inference data. Therefore, a total of 5 training inference data can be derived for each of Node 11, Node 12, Node 14, Node 15, and Node 16.

[0173] Additionally, a plurality of branch nodes included in the tree can be classified into nodes containing correct answers and nodes containing incorrect answers, and in FIG. 7, a plurality of branch nodes can be classified into branch nodes colored with hatching and branch nodes colored with dots. Preferably, branch nodes colored with hatching correspond to nodes containing incorrect answers, and branch nodes colored with dots correspond to nodes containing correct answers.

[0175] FIG. 8 schematically illustrates the process of performing a learning step according to one embodiment of the present invention.

[0177] In summary, FIG. 8 (a) illustrates the process of performing the step of training the LLM, and FIG. 8 (b) illustrates the process of performing the step of training the target model (3000).

[0179] Specifically, the learning step can train the LLM (2000) using the above-mentioned inference data for training, or train a target model (3000) other than the LLM (2000) using the above-mentioned inference data for training.

[0181] As illustrated in FIG. 8(a), the LLM (2000) can be trained using the derived inference data for training. At this time, the LLM (2000) corresponds to an LLM that derives an answer including a first answer and a second answer, and the LLM (2000) trained through the inference data for training can derive a more accurate answer than before training when a question including a task question is input thereafter, or can derive an answer containing higher quality inference content, or can automatically derive high quality inference data.

[0183] As illustrated in FIG. 8(b), a target model (3000) other than the LLM (2000) can be trained using the derived training inference data. At this time, the target model (3000) may be located outside or inside the server system (1000) and may be a model that includes the LLM. The target model (3000) trained through the training inference data can subsequently derive a more accurate answer than before training when a specific question is input, or derive an answer containing higher quality inference content, or automatically derive high-quality inference data.

[0185] Preferably, the learning inference data derived in the present invention is high-quality inference data including an inference process, inference content, and inference result, and can be used as data capable of training various LLMs outside or inside the server system (1000).

[0187] FIG. 9 illustrates, in an exemplary manner, the internal configuration of a computing device (11000) according to one embodiment of the present invention.

[0189] The server system (1000) mentioned in the description of FIG. 1 may include components of the computing device (11000) illustrated in FIG. 9, which will be described later.

[0191] As illustrated in FIG. 9, the computing device (11000) may include at least one processor (11100), memory (11200), peripheral interface (11300), input / output subsystem (I / O subsystem) (11400), power circuit (11500), and communication circuit (11600).

[0193] Specifically, the memory (11200) may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. The memory (11200) may include software modules, instruction sets, or various other data required for the operation of the computing device (11000).

[0194] At this time, access to the memory (11200) from other components, such as the processor (11100) or the peripheral device interface (11300), can be controlled by the processor (11100). The processor (11100) may be composed of a single or multiple units and may include processors in the form of GPUs and TPUs to improve computational processing speed.

[0195] The above peripheral device interface (11300) can connect input and / or output peripheral devices of the computing device (11000) to the processor (11100) and the memory (11200). The processor (11100) can perform various functions for the computing device (11000) and process data by executing a software module or instruction set stored in the memory (11200).

[0196] The input / output subsystem (11400) may connect various input / output peripheral devices to the peripheral device interface (11300). For example, the input / output subsystem (11400) may include a controller for connecting peripheral devices such as a monitor, keyboard, mouse, printer, or, if necessary, a touchscreen or sensor to the peripheral device interface (11300). According to another aspect, the input / output peripheral devices may be connected to the peripheral device interface (11300) without passing through the input / output subsystem (11400).

[0197] The power circuit (11500) may supply power to all or part of the components of the terminal. For example, the power circuit (11500) may include one or more power sources such as a power management system, a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or any other components for power generation, management, and distribution.

[0198] The communication circuit (11600) may enable communication with another computing device using at least one external port. Alternatively, as described above, the communication circuit (11600) may enable communication with another computing device by including an RF circuit and transmitting and receiving an RF signal, also known as an electromagnetic signal, as needed.

[0200] The embodiment of FIG. 9 is merely an example of the computing device (11000), and the computing device (11000) may have some components shown in FIG. 9 omitted, additional components not shown in FIG. 9 added, or a configuration or arrangement that combines two or more components. For example, a computing device for a communication terminal in a mobile environment may include a touchscreen or sensors in addition to the components shown in FIG. 9, and the communication circuit (1160) may include a circuit for RF communication of various communication methods (Wi-Fi, 3G, LTE, 5G, 6G, Bluetooth, NFC, Zigbee, etc.). The components that can be included in the computing device (11000) may be implemented as hardware, software, or a combination of both hardware and software, including one or more integrated circuits specialized for signal processing or applications.

[0201] Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded on a computer-readable medium. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. An application to which the present invention is applied may be installed on a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file upon a request from the user terminal.

[0203] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0204] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be standardized and stored or executed in a standardized manner on a networked computing device. Software and data may be stored on one or more computer-readable recording media.

[0205] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0207] According to one embodiment of the present invention, the effect of automatically generating training inference data using a task dataset and a connecting word can be achieved.

[0208] According to one embodiment of the present invention, the effect of self-learning LLM through learning inference data can be achieved.

[0209] According to one embodiment of the present invention, when the LLM derives an answer that is not the correct answer to an input task question, it can apply a connecting word to derive a modified answer, thereby achieving the effect of deriving a modified answer.

[0210] According to one embodiment of the present invention, when the LLM derives a correct answer to an input task question, it can produce the effect of deriving inference data for training.

[0211] According to one embodiment of the present invention, the effect of being able to train an LLM and a target model using training inference data can be achieved.

[0212] According to one embodiment of the present invention, a branching node can be generated by applying a connecting word to a non-correct answer among the answers derived from LLM.

[0213] According to one embodiment of the present invention, a tree including branch nodes is generated based on a plurality of connecting words, and the effect of deriving inference data for training based on the tree can be achieved.

[0214] According to one embodiment of the present invention, the effect of automatically generating inference data that satisfies diversity and quality using connecting words can be achieved.

[0215] According to one embodiment of the present invention, the effect of continuously developing a model with excellent inference performance through self-learning of the LLM can be achieved.

[0217] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents. Therefore, other implementations, other embodiments, and equivalents to the claims below are also within the scope of the claims.

Claims

Claim 1 A method for automatically generating training inference data using a connector executed in a server system comprising one or more processors and one or more memories, comprising: a task question extraction step of extracting a plurality of task questions from a task dataset including a task question and a task answer corresponding to the task question; a first answer derivation step of deriving a first answer by inputting the extracted task questions into an LLM outside or inside the server system; a first answer verification step of determining whether the first answer is a correct answer by comparing the first answer with the task answer corresponding to the extracted task question; a second answer derivation step of, if the first answer is not a correct answer, defining the task question and the first answer as an incorrect answer sample and a root node, and forming a tree by deriving second answers corresponding to a plurality of branch nodes having a first depth by applying each of a plurality of connectors pre-set to the incorrect answer sample and inputting them into the LLM; and determining whether the second answers are correct answers by comparing the second answers with the task answer, and the A second answer verification step in which, if there is a second answer that is correct among the second answers, the path from the node of the correct second answer to the root node is traced back to form a set comprising the incorrect answer sample, the first connector used in the path, and the second answer, and the second answer is derived as training inference data; a third answer derivation step in which, if there is a second answer that is not correct among the second answers, the second answer is set as the parent node and each of the plurality of connectors is applied to input into the LLM to derive third answers corresponding to branch nodes having a second depth, thereby expanding the tree; and a training step in which at least one of the LLM and a target model other than the LLM is trained using the training inference data.A method for automatically generating training inference data, comprising: a plurality of preset connectors, wherein the preset connectors include prompts that induce the LLM to derive at least one inference process among sequential expansion, re-evaluation, double-check, hypothesis conversion, and contradiction recognition; wherein the expansion of the tree is performed repeatedly until a preset depth is reached; and wherein the training inference data comprises a set of paths from a leaf node containing a correct answer among the leaf nodes of the tree to the root node. Claim 2 The method for automatically generating learning inference data according to claim 1 further comprises a third answer verification step of comparing the third answer with the task correct answer to determine whether the third answer is correct, and if the third answer is correct, deriving the learning inference data by configuring the incorrect answer sample, the first connecting word for the incorrect answer sample, the second answer, the second connecting word for the second answer, and the third answer into a set. Claim 3 delete Claim 4 The method of claim 1 for automatically generating the learning inference data comprises: a server system inputting each of a plurality of task questions included in the task dataset into an LLM to detect a task question from which a non-correct answer is derived; applying each of a plurality of pre-set connectors to each of the detected task questions to derive a modified answer; and based on the modified answer, classifying the plurality of connectors into connectors that derive a correct answer and connectors that derive a non-correct answer. Claim 5 delete Claim 6 delete Claim 7 A server system comprising one or more processors and one or more memories, and performing a method for automatically generating training inference data using connectors, comprising: a task question extraction unit that extracts a plurality of task questions from a task dataset including a task question and a task answer corresponding to the task question; a first answer derivation unit that inputs the extracted task questions into an LLM outside or inside the server system to derive a first answer; a first answer verification unit that compares the first answer with the extracted task answer corresponding to the task question to determine whether the first answer is correct; a second answer derivation unit that, if the first answer is not correct, defines the task question and the first answer as an incorrect answer sample and a root node, and forms a tree by deriving second answers corresponding to a plurality of branch nodes having a first depth by applying each of a plurality of pre-set connectors to the incorrect answer sample and inputting them into the LLM; and compares the second answers with the task answer to determine whether the second answers are correct, and A second answer verification unit that, when a second answer is correct among the second answers, traces back the path from the node of the correct second answer to the root node to form a set of the incorrect answer sample, the first connector used in the path, and the second answer, and derives it as inference data for training; a third answer derivation unit that, when a second answer is not correct among the second answers, sets the second answer as the parent node and applies each of the plurality of connectors to input it into the LLM to derive third answers corresponding to branch nodes having a second depth, thereby expanding the tree; and a learning unit that trains at least one of the LLM and a target model other than the LLM using the inference data for training.A server system comprising, wherein the plurality of preset connectors include prompts that induce the LLM to derive at least one inference process among sequential expansion, re-evaluation, double-check, hypothesis conversion, and contradiction recognition, the expansion of the tree is performed repeatedly until a preset depth is reached, and the training inference data comprises a set of paths from a leaf node containing the correct answer among the leaf nodes of the tree to the root node. Claim 8 A computer-readable storage medium for implementing a method for automatically generating training inference data using a connector executed in a server system comprising one or more processors and one or more memories, wherein the computer-readable storage medium comprises computer-executable instructions that cause the server system to perform the following steps, the following steps comprising: a task question extraction step of extracting a plurality of task questions from a task dataset including a task question and a task answer corresponding to the task question; a first answer derivation step of inputting the extracted task question into an LLM outside or inside the server system to derive a first answer; a first answer verification step of comparing the first answer with the extracted task answer corresponding to the task question to determine whether the first answer is a correct answer; and, if the first answer is not a correct answer, defining the task question and the first answer as an incorrect answer sample and a root node, and applying each of a plurality of pre-set connectors to the incorrect answer sample and inputting them into the LLM to correspond to a plurality of branch nodes having a first depth A second answer derivation step for deriving second answers and forming a tree; a second answer verification step for comparing the second answers with the task correct answer to determine whether the second answers are correct, and if there is a second answer that is correct among the second answers, tracing back the path from the node of the correct second answer to the root node to form a set of the incorrect answer sample, the first connector used in the path, and the second answer to derive inference data for training; a third answer derivation step for expanding the tree by deriving third answers corresponding to branch nodes having a second depth by inputting the second answer as a parent node and applying each of the plurality of connectors to the LLM, if there is a second answer that is not correct among the second answers; and a training step for training at least one of the LLM and a target model other than the LLM using the inference data for training.A computer-readable storage medium comprising: a plurality of preset connectors, wherein the preset connectors include prompts that induce the LLM to derive at least one inference process among sequential expansion, re-evaluation, double-check, hypothesis conversion, and contradiction recognition; wherein the expansion of the tree is performed repeatedly until a preset depth is reached; and wherein the training inference data comprises a set of paths from a leaf node containing a correct answer among the leaf nodes of the tree to the root node.

Images