Method and system for generating answers
The method and system for generating answers to queries using a pre-trained large language model with a stepwise thinking structure addresses the inconsistency in multi-step reasoning, enhancing reliability and accuracy by maintaining a consistent thought process and correcting errors.
Patent Information
- Application Number
- PCT/KR2026/095402
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2026-03-18
- Filing Date
- 2026-04-01
- Publication Date
- 2026-11-12
AI Technical Summary
Large language models struggle to consistently perform multi-step reasoning for complex queries, often omitting intermediate thought processes and failing to maintain a stable stepwise thought process.
A method and system for generating answers to queries using a pre-trained large language model, structured through a stepwise thinking process, with self-correction and interactive modification capabilities, ensuring each reasoning step is maintained and errors are detected and corrected.
Enhances the reliability and accuracy of generating answers to complex queries by maintaining a consistent stepwise thinking process, improving inference performance and reducing errors.
Smart Images

Figure KR2026095402_12112026_PF_FP_ABST
Abstract
Description
Answer generation method and system
[0001] The present invention relates to an answer generation method and system capable of generating an answer to a query. Furthermore, the present invention relates to a training method and system for a large language model that generates an answer to a query.
[0002] The dictionary definition of artificial intelligence is a technology that realizes human learning, reasoning, perception, and natural language understanding abilities through computer programs. With the advancement of deep learning, the ability of such artificial intelligence to automatically recognize and learn complex patterns has improved, and it is being widely utilized in various industrial fields.
[0003] In particular, driven by the advancement of artificial intelligence, various language models have been developed. These models have reached a level where they not only recognize text and understand its meaning but also extract and classify information from vast amounts of text-based data, such as documents, and even generate text directly.
[0004] These language models are being actively utilized in various fields. For example, they are used in diverse areas where text-based tasks can be performed, such as search engines, document creation (e.g., resume writing, report writing, posting, etc.), free conversation on various topics, data parsing from given text (e.g., data summarization, classification, etc.), provision of expertise, programming, and converting given sentences into sentences of an appropriate style.
[0005] In this regard, Large Language Models (LLMs) have recently emerged, which understand and generate human language through prior training on vast amounts of text data. Unlike chatbots, which are manually built and provide only limited answers, LLMs are demonstrating innovation in the artificial intelligence market through their technological capabilities to communicate naturally, almost like humans, and provide fast and accurate information.
[0006] These large language models are evolving to handle tasks ranging from generating answers to simple queries to long-term reasoning that requires deriving answers through multiple inference steps. In this context, when generating answers to high-difficulty queries involving multiple sequential thought stages, large language models are required to perform relatively more complex reasoning compared to generating answers in a single step. For example, in problems involving multiple decision-making stages—such as mathematical problem-solving, logical judgment, and legal analysis—it is essential to generate answers based on a step-by-step thought structure during the problem-solving process.
[0007] However, large language models trained on insufficient training data to generate answers to queries through a stepwise thinking structure may tend to fail to stably form such stepwise thought processes. In particular, for queries requiring multi-step reasoning, it may be observed that the development of each reasoning step in the large language model is not consistently maintained, or that intermediate thought processes are omitted or not sufficiently considered.
[0008] Accordingly, there is a need for technology that trains large language models to more clearly form step-by-step thought processes during the query answer generation process, and enables large language models to generate answers to queries according to step-by-step thought structures.
[0009] The present invention is intended to provide an answer generation method and system capable of generating an answer to a query according to a stepwise thinking structure.
[0010] Specifically, the present invention aims to provide an answer generation method and system that enables a large language model to generate an answer through a logical thinking process for a query requiring multiple thinking steps.
[0011] Furthermore, the present invention aims to provide an answer generation method and system capable of generating customized answers to a query by controlling a large language model so that an appropriate thought structure is applied according to the characteristics of the query.
[0012] In addition, the present invention aims to provide a method and system for training a large language model capable of training the large language model so that the generation of answers to queries is performed according to a step-by-step thinking structure.
[0013] Furthermore, the present invention aims to provide a training method and system for a large language model capable of generating training data configured such that the generation of answers to queries is performed according to a step-by-step thought structure.
[0014] In addition, the present invention is intended to provide a self-correction reasoning function that automatically detects errors occurring during the reasoning process and switches the direction of reasoning.
[0015] Furthermore, the present invention is intended to provide an interactive answer generation function that allows a user to directly modify inference items and regenerate an answer based thereon.
[0016] To solve the problem described above, the present invention proposes a method and system for providing an answer to a query using a pre-trained large language model such that the generation of an answer to a query is performed according to a stepwise inference flow. The answer generation method according to the present invention, which is performed by a computer, may include the steps of receiving a query, generating thought structure data having at least one inference item for the query, obtaining ground data corresponding to the at least one inference item, providing the ground data and the query as inputs to a pre-trained large language model to obtain an answer to the query from the large language model, and providing the answer to a user terminal.
[0017] In one embodiment, the accident structure data is generated based on an inference step sequence for the query, and the inference step sequence may include at least one of a problem analysis step, an access setting step, a calculation step, and a verification step.
[0018] In one embodiment, the step of acquiring the basis data may generate the basis data corresponding to each of the plurality of inference items for each inference step sequence.
[0019] In one embodiment, the step of acquiring the basis data may sequentially generate the basis data according to a predetermined order of at least one inference item constituting the accident structure data.
[0020] In one embodiment, the method further includes the step of generating thought process data for the query using the basis data, and in the step of generating the thought process data, the generated basis data can be sequentially combined to generate the thought process data.
[0021] In one embodiment, the thought process data includes the basis data and the answer data to the query generated based on the basis data, and in the step of obtaining the answer to the query from the large language model, at least a portion of the thought process data can be obtained as the answer to the query.
[0022] In one embodiment, in the step of generating the accident structure data, the accident structure data may be generated to include at least one inference item related to the domain data based on the domain data corresponding to the query.
[0023] In one embodiment, in the step of providing the answer to the user terminal, the at least one inference item and the basis data corresponding to the at least one inference item may be provided as an answer to the query through at least one service page.
[0024] In one embodiment, the answer generation method of the present invention may further include the steps of: generating the modification thought structure data such that, when a modification event for a specific inference item is received from the user terminal through the service page, the modification inference item is modified according to the modification event; providing the generated modification thought structure data as input to the pre-trained large language model; generating modification basis data corresponding to the at least one modification inference item in the large language model; receiving a modification answer generated based on the modification basis data from the large language model; and providing the modification answer for the query to the user terminal.
[0025] An answer generation system according to the present invention includes a memory for storing instructions and at least one processor electrically connected to the memory, and when the instructions are executed by the at least one processor, the at least one processor receives a query from a user terminal, generates thought structure data having at least one inference item for the query, obtains basis data corresponding to the at least one inference item, provides the basis data and the query as input to a pre-trained large language model, obtains an answer to the query from the large language model, and provides the answer to the user terminal.
[0026] A method for training a large language model according to the present invention, performed by a computer, may include the steps of: specifying a reference learning data set corresponding to at least one query; generating learning thought structure data for the query using the reference learning data set; obtaining learning basis data corresponding to at least one inference item constituting the learning thought structure data; generating a learning data set for training a large language model based on the learning basis data; and training the large language model to generate an answer to a user query using the learning data set.
[0027] In one embodiment, in the step of acquiring the learning basis data, the learning basis data can be generated sequentially according to the order pre-set in the at least one inference item constituting the learning thought structure data.
[0028] In one embodiment, the learning method of the large language model further includes the step of generating learning thought process data for the query using the learning basis data, and in the step of generating the learning thought process data, the generated at least one learning basis data may be sequentially combined to generate the learning thought process data.
[0029] In one embodiment, the learning thought process data is configured to include the learning basis data and the learning answer data for the at least one query, and the learning data set may be configured to include the at least one query and the learning thought process data corresponding to each of the at least one query.
[0030] In one embodiment, the step of specifying the reference learning data set may include: specifying candidate query data corresponding to at least one query among a plurality of basic learning data according to a preset similarity criterion; calculating domain similarity between domain data corresponding to the at least one query and candidate domain data corresponding to the candidate query data; identifying candidate inference scale data corresponding to the candidate query data and calculating inference scale similarity between the candidate inference scale data; and specifying reference query data among the candidate query data based on at least one of the domain similarity and the inference scale similarity, and specifying a reference learning data set including at least one of reference inference flow data and reference inference scale data corresponding to the reference query data.
[0031] In one embodiment, the step of generating the learning thought structure data may include providing the reference query data, the reference thought structure data, the reference inference scale data, and the at least one query as inputs to a pre-trained reference large language model, and generating the learning thought structure data for the at least one query in the reference large language model based on the reference learning data set.
[0032] In one embodiment, in the step of generating the learning thought structure data, the number of at least one inference item can be predicted using at least one of the reference thought structure data and the reference inference scale data, and the learning thought structure data can be generated based on the predicted number of inference items.
[0033] In one embodiment, a prompt requesting the generation of the learning basis data is generated using the at least one query, the pre-generated learning basis data, the first inference item, and the second inference item, and the prompt is provided as input to a pre-trained reference large language model together with the learning thought structure data, and the learning basis data can be generated by referring to the second inference item in order to generate the first learning basis data corresponding to the first inference item based on the prompt in the reference large language model.
[0034] In one embodiment, the step of generating the learning data set may include comparing the learning answer data included in the learning thinking process data with the correct answer data corresponding to the at least one query, and generating the learning data set to include specific learning thinking process data satisfying a preset filtering condition and specific queries corresponding to the specific learning thinking process data based on the comparison result.
[0035] A learning system for a large language model according to the present invention includes a memory for storing instructions and at least one processor electrically connected to the memory. When the instructions are executed by the at least one processor, the at least one processor identifies a reference learning data set corresponding to at least one query, generates learning thought structure data for the query using the reference learning data set, obtains learning basis data corresponding to at least one inference item constituting the learning thought structure data, generates a learning data set for learning a large language model based on the learning basis data, and can train the large language model to generate an answer to a user query using the learning data set.
[0036] As seen above, according to the present invention, an answer to a query can be generated according to a stepwise thinking structure. Through this, multiple thinking steps performed during the answer generation process can be structurally unfolded, and by maintaining a consistent thinking process according to the stepwise thinking structure, the reliability and inference performance of generating answers to complex queries can be improved.
[0037] Furthermore, according to the present invention, a large language model can generate an answer to a query requiring multiple thinking stages through a step-by-step thinking process. Through this, even for complex queries, an answer can be generated by reasoning step-by-step, thereby expanding the problem-solving scope and application possibilities of the large language model.
[0038] Furthermore, according to the present invention, an appropriate thinking structure can be controlled to be applied according to the characteristics of the query. Through this, by selectively performing a suitable thinking process according to the domain of the query or the required scale of inference, it is possible to prevent situations where unnecessary excessive inference is performed or insufficient inference is not achieved, and to improve the efficiency and accuracy of generating customized answers to the query.
[0039] Furthermore, according to the present invention, a large language model can be trained using training data configured to generate answers according to a stepwise thinking structure. Through this, the large language model can effectively learn stepwise thinking process patterns during the answer generation process, thereby improving long-term reasoning ability for complex queries and consistency in response generation.
[0040] Furthermore, according to the present invention, training data can be generated configured such that the generation of answers to queries is performed according to a step-by-step thinking structure. Through this, training data including a step-by-step thinking process can be efficiently constructed, and by ensuring diversity and scalability of training data for various problem types and domains, the generalization performance of large language models and their usability in actual service environments can be improved.
[0041] Furthermore, according to the present invention, by applying a processing method based on an inference step sequence to break down the query resolution process into multiple stages, higher inference accuracy can be provided compared to a single-stage inference method. In addition, this stage-based processing structure can reduce the possibility of errors occurring when a large language model processes complex queries.
[0042] Furthermore, according to the present invention, by systematically selecting a reference training dataset based on domain similarity and inference scale similarity, a high-quality training dataset containing an inference flow structure optimized for the characteristics of each query can be automatically generated. This allows for the improvement of inference quality in large language models while reducing the cost and time required to build training data.
[0043] In addition, according to the present invention, consistent reasoning data that considers the subsequent reasoning flow can be generated at each reasoning stage through a look-ahead reasoning generation method that refers to the next reasoning item. This maintains logical continuity between reasoning stages and improves the consistency of the final answer.
[0044] FIG. 1 illustrates an example of a block diagram of a computing system in which the present invention can be implemented.
[0045] FIG. 2 illustrates an example of a block diagram of a computing device that may be included in a user computing device, a server computing system, and a training computing system, as an embodiment of a computing system in which the present invention can be implemented.
[0046] Figure 3 illustrates an example of a block diagram from another perspective of a computing device, which is one of the components of a computing system.
[0047] FIG. 4 is a conceptual diagram illustrating an answer generation system and a learning system for a large language model according to the present invention.
[0048] FIGS. 5A and 5B are conceptual diagrams for explaining the answer generation process and the large language model learning process in general according to the present invention.
[0049] FIG. 6 is a flowchart illustrating a method for generating an answer according to the present invention.
[0050] FIGS. 7a to 7d and FIGS. 8a to 8c are conceptual diagrams for explaining a learning method for a large language model according to the present invention.
[0051] FIGS. 9a to 9c are conceptual diagrams for explaining a method for generating answers using a large language model according to the present invention.
[0052] FIGS. 10a to 10c are conceptual diagrams for explaining embodiments of providing answers to queries generated by a large language model according to the present invention.
[0053] FIG. 11 is a conceptual diagram illustrating the performance of a pre-trained large language model according to the present invention.
[0054] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components are assigned the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not have distinct meanings or roles in themselves. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention.
[0055] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.
[0056] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.
[0057] Singular expressions include plural expressions unless the context clearly indicates otherwise.
[0058] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.
[0059] Hereinafter, the present invention will be examined in more detail with reference to the attached drawings. FIG. 1 illustrates an example of a block diagram of a computing system in which the present invention can be implemented. FIG. 2 illustrates an example of a block diagram of a computing device that may be included in a user computing device, a server computing system, and a training computing system as an embodiment of a computing system in which the present invention can be implemented. FIG. 3 illustrates an example of a block diagram of a computing device in another aspect that is one of the components of a computing system.
[0060] Meanwhile, FIG. 1 illustrates an example of a block diagram of a computing system in which the present invention can be implemented. In this regard, the answer generation system and the learning system of a large language model according to the present invention can be implemented through a computing device described below and can perform data processing related to the aforementioned answer generation method and the learning method of a large language model.
[0061] Referring to FIG. 1, a computing system (10000) for performing the answer generation method and the learning method of a large language model of the present invention may include at least one computing device. At this time, the at least one computing device may be a single processor or a multiprocessor computing device.
[0062] The components of at least one computing device of the present invention may include at least one processor, memory, other hardware, and various hardware components such as a system bus (not shown) that connects various system components to transmit and receive data to and from each other (e.g., telecommunicationally connected, physically connected, electrically connected), and the components of at least one computing device are not limited thereto and may be very diverse.
[0063] Meanwhile, at least one computing device included in the computing system (10000) that performs the answer generation method and the large language model learning method may be connected to communicate via a network (1070). For example, at least one computing device included in the computing system (10000) may be clustered or may be part of a local area network (LAN). Additionally, at least one computing device may be part of a wide area network (WAN) or connected to at least one of a client-server network and a peer-to-peer network within the cloud.
[0064] Meanwhile, when at least one computing device is used in at least one of a network environment and a cloud computing environment, the at least one computing device may be connected to at least one of a public and a private network through a network interface or an adapter. In one embodiment, other communication connection devices, such as a modem, may be used to establish communication through the network. The modem may be at least one of an internal modem and an external modem, and may be connected to a system bus through a network interface or a specific mechanism, etc. A wireless network component consisting of an interface and an antenna may be coupled to the network through a device such as an access point, a peer computer, etc. In the present invention, the method by which at least one computing device is connected to communicate through the network (1070) is not limited and may be connected to communicate in a manner different from the described examples.
[0065] Furthermore, other computer-type devices or systems not shown in FIG. 1 may also interact technically with at least one computing device or other system through one or more connections to the network (1070) via a network interface. Here, the network interface may include network interface equipment such as a physical network interface controller (NIC) or a virtual network interface (VIF).
[0066] The network (1070) of the present invention may include various forms such as the Internet, WLAN (Wireless LAN), Wi-Fi (Wireless Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wireless USB (Wireless Universal Serial Bus), etc., and in the present invention, data transmission may be performed based on standard communication protocols such as TCP / IP, HTTP, SSL, etc.
[0067] A computing system (10000) that performs the answer generation method and the large language model learning method according to the present invention may include at least one of a user computing device (1010), a training computing system (1050), and a server computing system (1030).
[0068] A user computing device (1010) according to the present invention may be understood as a computing device comprising at least one processor (1011) and a memory (1012) for performing an answer generation method and a learning method for a large language model. For example, the user computing device (1010) may include at least one computing device among a smartphone, a smart TV, a laptop computer, a desktop computer, a digital broadcasting terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, a slate PC, a tablet PC, an ultrabook, a wearable device (e.g., a smartwatch, a smart glass, and a head-mounted display).
[0069] At least one processor (1011) constituting the user computing device (1010) may include one or more general-purpose processors or one or more special-purpose processors. For example, at least one processor (1011) constituting the user computing device (1010) may be composed of at least one of a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), an arithmetic logic unit (ALU), a floating-point arithmetic unit (FPU), an application integrated circuit, an application semiconductor (ASIC), digital signal processing devices (DSPs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electrical units for performing functions, or a plurality of electrically connected processors.
[0070] Furthermore, at least one processor (1011) may be configured to execute a computer-readable instruction contained in memory (1012) or other instructions described herein.
[0071] The memory (1012) constituting the user computing device (1010) according to the present invention may include volatile memory, non-volatile memory, fixed media, removable media, magnetic media, optical media, semiconductor media, or other types of physically durable storage media.
[0072] For example, memory (1012) may include one or more non-transient or transient computer-readable storage media such as RAM, ROM, HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), EEPROM, EPROM, flash memory device, magnetic disk, etc., and may include web storage of a server that performs the storage function of memory on the internet. Such memory (1012) may store data and instructions necessary for the at least one processor (1011) to train (or optimize, fine-tune) a large language model and to perform the operation of an application to provide an answer to a query using the pre-trained large language model.
[0073] A user computing device (1010) may include one or more user input components (1021) that detect user input. For example, the user input component (1021) may also be referred to as a user interface module. The user input component (1021) may include a touch screen, a computer mouse, a keyboard, a keypad, a touchpad, a trackball, a joystick, a voice recognition module, or other similar devices. However, the present invention does not limit the type of user input component (1021). In this case, the user input component (1021) in the present invention does not necessarily mean a hardware means, but can be understood as a channel for receiving input from a user. Meanwhile, the user of the present invention may refer to an automated agent, script, playback software, etc., that operates on behalf of one or more people.
[0074] A user can interact with a computing system (10000) including at least one computing device through input text, touch, voice, movement, computer vision, gesture, or other forms of input or output using a user input component (1021). For example, the user input component (1021) may include one or more of a command line interface (CLI), a graphical user interface (GUI), a natural user interface (NUI), a voice command interface, or other user interface (UI) representations.
[0075] Between the user input component (1021) and the user computing device (1010), one or more application programming interface (API) calls may be made based on user input received from a user interface or network.
[0076] Here, the expression "based on" may be interpreted to include cases where it is based on the use of a specific configuration, modified from, derived from, influenced by, dependent on, or otherwise derived from a specific configuration. In some embodiments, an API call may be configured for a specific API, which may be interpreted or converted into an API call configured for another API. Here, an API may refer to a defined interface or connection between computers or between computer programs.
[0077] In one embodiment, the user computing device (1010) may store at least one machine learning model (1020). For example, the user computing device (1010) may be various machine learning models, such as a plurality of neural networks (e.g., deep neural networks) that perform at least one of generating answers to queries and learning large language models, or other types of machine learning models including non-linear models or linear models, and may be composed of a combination thereof.
[0078] According to an embodiment of the present invention, a user computing device (1010) may perform an answer generation method and a large language model learning method using a local or external machine learning model (1020). Alternatively, the user computing device (1010) may perform an answer generation method and a large language model learning method using a machine learning model (1040) provided by a server.
[0079] In addition, according to another embodiment of the present invention, a server computing system (1030) communicating with a user computing device (1010) may provide an answer to a query according to the present invention to the user computing device (1010) via an application or the web in accordance with a request from a user received through the user computing device (1010).
[0080] In addition, according to another embodiment of the present invention, by performing the answer generation method and the large language model learning method of the user computing device (1010) and the server computing system (1030), an answer to a query generated using a pre-trained large language model can be provided to the user.
[0081] Additionally, according to various embodiments of the present invention, a user computing device (1010) or a server computing system (1030) can learn machine learning models (1020, 1040) performed in an answer generation method and a large language model learning method through interaction with a training computing system (1050) that is communicatedly connected via a network (1070). In this case, the training computing system (1050) may be a computing system separate from the server computing system (1030). Alternatively, in some embodiments, the training computing system (1050) may be part of the server computing system (1030) or part of the user computing device (1010).
[0082] Meanwhile, the server computing system (1030) may include at least one processor (1031) and memory (1032). Here, the processor (1031) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), an application integrated circuit, an application semiconductor (ASIC), an arithmetic logic unit (ALU), a floating-point arithmetic unit (FPU), digital signal processing devices (DSPs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electrical units for performing functions. For example, at least one processor (1031) may include a circuit and a transistor configured to execute instructions from memory (1032).
[0083] The memory (1032) constituting the server computing system (1030) according to the present invention may include volatile memory, non-volatile memory, fixed media, removable media, magnetic media, optical media, semiconductor media, or other types of physically durable storage media. For example, the memory (1032) may include one or more non-transient or transient computer-readable storage media such as RAM, ROM, HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), EEPROM, EPROM, flash memory device, magnetic disk, etc., and combinations thereof, and may include web storage of a server that performs the storage function of memory over the internet. Additionally, the server computing system (1030) may further include a data storage (data store). For example, the data storage may be composed of at least one of a relational database, a NoSQL database, a data warehouse, and a local file system.
[0084] In the memory (1032) constituting the server computing system (1030) according to the present invention, data and instructions necessary for the at least one processor (1031) to perform the operation of an application for at least one of generating an answer to a query and learning a large language model may be stored.
[0085] In one embodiment, the server computing system (1030) may be composed of a single device or a plurality of computing devices, and may be configured to operate according to a sequential architecture or a parallel computing architecture. Additionally, a distributed processing system may be configured with a plurality of networked devices.
[0086] Meanwhile, the training computing system (1050) may include at least one processor (1051) and memory (1052). The model trainer (1060) is a logical component that executes the training of at least one machine learning model (1020, 1040) and may be implemented in the form of hardware, firmware, or software. For example, the model trainer (1060) may be executed by the processor (1051) after loading training data (1061) stored in a storage device into memory (1052). For example, the model trainer (1060) may be configured to execute one or more operations (e.g., model training, model reconstruction, model validation, model testing) on at least one machine learning model.
[0087] The machine learning model of the present invention may include at least one of a statistical model, an algorithm, a neural network (NN), a convolutional neural network (CNN), a generative neural network(s), a Word2Vec model, a Bag of Words model, a TF-IDF (document frequency-inverse document frequency) model, a GPT (Generative Pre-trained Transformer) model (or other autoregressive models), a PPO (Proximal Policy Optimization) model, a nearest neighbor model (Ex: k-nearest neighbor model), a linear regression model, a K-means clustering model, a Q-learning model, a TD (Temporal Difference) model, a Deep Adversarial Network model, and all other types of models further described herein.
[0088] Specifically, the model trainer (1060) may execute operations to train a machine learning model, and said operations may include at least one of adding, removing, and modifying model parameters. At this time, the training of the machine learning model may be at least one of supervised learning, semi-supervised learning, and unsupervised learning. In one embodiment, the training of the machine learning model may include the step of repeatedly inputting training data (1061) based on epochs and repeatedly performing the machine learning model training process configured in this way. Here, an epoch may refer to a unit in which the entire set of training data (1061) undergoes forward and backpropagation processing once. In some implementations, different levels of training methods (e.g., supervised learning, semi-supervised learning, unsupervised learning) may be used for different epochs.
[0089] The training data (1061) of the present invention may include input data or data previously output from at least one machine learning model (e.g., recursive learning feedback).
[0090] At least one parameter of a machine learning model may include at least one of a seed value, a model node, a model layer, an algorithm, a function, connections between different machine learning models, connections between parameters, machine learning model constraints, and other digital components that influence the output of the machine learning model. In this case, model connections between different machine learning models may include or represent model parameters or relationships between models, and these may be dependent or interdependent, hierarchical, static, or dynamic. The combinations and configurations of model parameters described herein may be too complex to be maintained or utilized by human cognitive abilities.
[0091] In the present invention, the machine learning parameters described according to the embodiments are not limited, and a single machine learning model may further include a plurality of model parameters.
[0092] Meanwhile, FIG. 2 illustrates an example of a block diagram of a computing device (1100) that may be included in a user computing device (1010), a server computing system (1030), and a training computing system (1050), as an embodiment of a computing system (10000) in which the present invention can be implemented.
[0093] As illustrated in FIG. 2, the computing device (1100) may include at least one application (e.g., Application 1 to Application N), and each of the at least one application may include a machine learning library and a model execution environment for performing a machine learning-based answer generation method and a large language model training method. The at least one application included in the computing device (1100) may communicate with the sensor, context manager, device state manager, or additional component(s) within the computing device (1100) via an Application Programming Interface (API). In one embodiment, the at least one application may interface with device components, such as receiving sensor data or state data or transmitting prediction results to an output device via a public or private API.
[0094] Meanwhile, FIG. 3 illustrates an example of a block diagram in another aspect of a computing device (1200), which is one of the components of a computing system (10000) that performs an answer generation method and a learning method of a large language model according to an embodiment of the present invention.
[0095] A computing device (1200) according to the present invention may include at least one application (e.g., Application 1 to Application N), and at least one application may communicate with a central intelligence layer (1210). Each application may interact with a shared model within the central intelligence layer (1210) through an API (e.g., Common API).
[0096] The central intelligence layer (1210) includes one or more machine learning models and may share them among multiple applications or provide them independently to each. In one embodiment, the central intelligence layer (1210) may be integrated as part of an operating system or implemented as a separate logical layer.
[0097] Additionally, the central intelligence layer (1210) can communicate with the central device data layer (1220). The central device data layer (1220) can integrate and store a training data set stored within the computing device (1200) and provide it as input data necessary for training (e.g., fine-tuning) a large language model to be trained. Each device component (e.g., sensor, state manager, etc.) can communicate with the central device data layer (1220) via a private API, etc.
[0098] The technology described herein may be composed of a single or multiple computing devices, and the machine learning model performing the answer generation method and the large language model training method may be executed sequentially or in parallel on one component or multiple distributed components. The data storage, machine learning model, and application may be distributed and operated locally or over a network, and these configurations can be flexibly applied to various system architectures.
[0099] In the foregoing, a computing system capable of implementing the answer generation method and the large language model learning method according to the present invention has been described. Below, the answer generation method and the large language model learning method according to the present invention will be described in general.
[0100] Meanwhile, the answer generation system (100) and the learning system according to the present invention can also operate in an edge computing environment. Specifically, at least some functions of the answer generation system (100) can be performed using a lightweight large language model mounted on the user terminal (10) in an on-device manner, thereby enabling real-time answer generation without network delay. In this case, the lightweight on-device model can optimize memory and computation efficiency through techniques such as LoRA (Low-Rank Adaptation), quantization, and knowledge distillation.
[0101] In addition, the system according to the present invention may also be implemented as a multi-agent architecture. For example, a distributed inference structure may be formed in which multiple large language models process each inference item in parallel, and each agent may operate in a specialized structure dedicated to a specific domain or type of inference. In such a multi-agent structure, a control unit (130) acting as an orchestrator may integrate the inference results of each agent to generate a final answer.
[0102] Furthermore, the system according to the present invention can also be applied in a conversational environment. In a conversation that proceeds in multiple turns, the control unit (130) maintains the question-answer history of the previous conversation in the storage unit (120) as a memory context and references it when generating inference flow data for the current question, thereby generating a consistent answer that reflects the context of the conversation. At this time, the memory context may include at least one of a summary of the entire conversation history, key inference items, or intermediate answer data.
[0103] The present invention relates to a method and system for generating answers to a query according to a stepwise thinking structure using at least one artificial intelligence model. Here, the stepwise thinking structure may refer to structural information in which a plurality of thinking stages performed to generate an answer to a query are organized in a specific order (or relationship). For example, the stepwise thinking structure may also be understood as a thought flow (or inference flow) consisting of steps such as understanding the problem regarding the query, analyzing relevant information, a judgment process for problem solving, and deriving and verifying results.
[0104] Specifically, the present invention can generate answers to queries according to a stepwise thinking structure using a Large Language Model (LLM). The Large Language Model according to the present invention (or Large Language Model) may refer to a generative artificial intelligence model that is trained based on large-scale training data to understand and generate natural language, and can perform question answering, text generation, and reasoning for problem solving.
[0105] In the present invention, a large language model can be trained to generate answers to queries according to a stepwise thinking structure. At this time, "training a large language model" may refer to a process of adjusting the training parameters of a large language model by retraining, fine-tuning, or additionally training some or all of the parameters of the large language model using a given training dataset, so that the pre-trained large language model generates answers to queries according to the inference flow.
[0106] Specifically, the present invention can construct (or create, configure) a training dataset for training a large language model so that the generation of answers to queries is performed according to a stepwise thought structure. Furthermore, the present invention can use the generated training dataset to train a large language model to generate answers to queries according to a stepwise thought structure.
[0107] Specifically, in the present invention, a pre-trained large language model can generate an answer to a query based on thought structure data related to the generation of an answer to a query. Here, the thought structure data may refer to data representing a stepwise thought structure performed to generate an answer to a query. For example, the thought structure data may include a prompt template, a memory context for referencing information generated during a past query-answer process, reasoning tokens generated in token units during the stepwise thought process, or a combination thereof.
[0108] Meanwhile, in the present invention, the data constituting the thought structure data may be referred to as reasoning flow data. Here, reasoning flow data may refer to data in which at least one reasoning item, performed to generate an answer to a query, is configured according to a preset order. In this case, the reasoning item may refer to individual reasoning unit information (e.g., table of contents, outline) performed to derive an answer to a query.
[0109] Accordingly, in the present invention, the thought structure data may include structural information representing a thought structure performed to derive an answer to a query, and the thought structure data may be used as a concept that includes the inference flow data described in this specification. Accordingly, in this specification, thought structure data and inference flow data may be used with the same or similar meanings, and below, the thought structure data will be referred to as inference flow data for description.
[0110] Meanwhile, the pre-set order in at least one inference item constituting the inference flow data according to the present invention may refer to the logical processing order of a thought process performed to generate an answer to a query. For example, the pre-set order may include a logical flow in which a step of understanding a problem included in the query, an analysis step for problem solving, a step of performing a solution process, and a step of deriving or verifying a result are arranged sequentially.
[0111] Furthermore, the pre-set order according to the present invention is not limited to a specific sequential structure and can be configured in various forms depending on the type of query or domain. For example, the inference items may be configured in a stepwise structure arranged sequentially, or they may be configured in a structure where some inference items are executed in parallel or selectively according to specific conditions. Accordingly, the pre-set order of the inference items according to the present invention can be configured in various forms and is not limited to a specific inference structure or processing order.
[0112] In the present invention, supporting data corresponding to inference items constituting inference flow data is obtained from a large language model or at least one external server, and an answer to a query can be generated through a step-by-step thought process based on the obtained supporting data.
[0113] The basis data according to the present invention may refer to information generated or acquired in correspondence with each inference item in a stepwise thinking process for deriving an answer to a query. Specifically, the basis data may include information corresponding to at least one of inference, judgment, analysis, calculation, comparison, verification, or explanation performed in the process of generating an answer to a query.
[0114] For example, the basis data may include at least one of intermediate judgment results, analysis results, explanatory information, and calculation results generated by a large language model, and may also include document information, knowledge information, reference information, or database query results retrieved or acquired from an external server. Accordingly, the basis data according to the present invention can be understood as data including intermediate information generated or acquired in a stepwise thought process for generating an answer to a query, or information serving as the basis for the thought process.
[0115] Meanwhile, the basis data described in the present invention may be configured to include inference data. That is, the basis data in the present invention may be used with the same or similar meaning as inference data that includes judgment basis or logical basis information generated or acquired stepwise during the answer generation process. Accordingly, in this specification, basis data may be named or understood as inference data depending on the context, and hereinafter, basis data will be described by naming it as inference data.
[0116] In the present invention, inference flow data and inference data corresponding to at least one inference item constituting the inference flow data can be generated. Here, the thought process data is data representing a series of stepwise thought processes performed to generate an answer to a query, and may refer to data composed by combining (or connecting) at least one inference data and the answer data to the query.
[0117] Such thought process data may be configured to include reasoning process data. That is, the reasoning process data in the present invention is data representing a step-by-step decision process or thought flow for solving a query, and may be used with the same or similar meaning as thought process data. Accordingly, in this specification, thought process data may be named or understood as reasoning process data depending on the context, and for convenience of explanation, thought process data is referred to as reasoning process data in the following description.
[0118] Meanwhile, the answer generation system and the learning system of a large language model according to the present invention may include at least one artificial intelligence model. In this case, the at least one artificial intelligence model mentioned in the present invention may include various artificial intelligence models that can be utilized depending on various situations or purposes. For example, the artificial intelligence model of the present invention may include at least one of a Large Language Model (LLM), a super-large Foundation model, a Generative AI model, a Vision Language Model (VLM), a Machine Learning (ML) model, a Deep Learning model, a Deep Neural Network (DNN), a Transformer-based model, a Supervised Learning (SL) model, a Reinforcement Learning (RL) model, and a special purpose model (e.g., a time series forecasting model (e.g., ARIMA model, SARIMA model, etc.), a time series Foundation model, a Graph Neural Network, a Multimodal Model, a Natural Language Processing (NLP) model, a computer vision model, a speech recognition model, a synthesis model, a recommendation system model, etc.).
[0119] Furthermore, the artificial intelligence model used in the answer generation system and the learning system of a large language model according to the present invention may be implemented with a single or multiple components. More specifically, the artificial intelligence model used in the present invention may be implemented with a single component (1) or with multiple components (2, 3, 4 or more, etc.).
[0120] In this regard, when multiple artificial intelligence models are implemented in the present invention, the multiple artificial intelligence models may be implemented as models of the same type. In this case, the multiple artificial intelligence models may be implemented to perform different functions (or roles) in the present invention.
[0121] In the foregoing, the answer generation method and the learning method of a large language model according to the present invention have been generally described, and this can be implemented by the answer generation system and the learning system of a large language model described below. Below, the answer generation system and the learning system of a large language model according to the present invention will be described in detail with reference to FIGS. 4, 5a, and 5b. FIG. 4 is a conceptual diagram for explaining the answer generation system and the learning system of a large language model according to the present invention, and FIGS. 5a and 5b are conceptual diagrams for generally explaining the answer generation process and the learning process of a large language model according to the present invention.
[0122] Meanwhile, as illustrated in FIG. 4, the answer generation system (100) according to the present invention may include at least one of a communication unit (110), a storage unit (120), a control unit (130), and a learning unit (140). However, the components of the answer generation system (100) according to the present invention are not limited thereto and may further include various hardware components that perform the same or similar roles as described in the description of the present specification.
[0123] Although not illustrated, the answer generation system (100) and the learning system of a large language model according to the present invention (hereinafter referred to as the “learning system” and described, not illustrated) may include at least one processor, and such processor may include one or more general-purpose processors or one or more special-purpose processors (e.g., digital signal processor, tensor processing unit (TPU), graphics processing unit (GPU), neural network processing unit (NPU), application integrated circuit, application semiconductor (ASIC), field programmable gate array (FPGA), quantum processing unit (or quantum processor, QPU), etc.). At least one processor may be configured to execute instructions, computer-readable instructions, or other instructions described herein that are stored (or included) in the storage unit (120).
[0124] In the answer generation system (100) and the learning system of a large language model according to the present invention, a memory and at least one processor can cooperate to perform the data processing described below. The processor can perform a series of operations and data processing using data and information stored in the memory. At this time, the memory may be a component of the storage unit (120).
[0125] In addition, the answer generation system (100) and the learning system according to the present invention can perform data processing and computation processes using quantum gates, quantum entanglement, and quantum superposition states, taking into consideration implementation in a quantum computer environment. For example, the present invention can perform parallel computations based on qubits, and such quantum computations can operate complementarily with existing classical computers.
[0126] Such quantum computers may include parallel computation using qubits and high-speed data processing devices utilizing quantum entanglement, and hardware-based computational optimization using FPGAs and ASICs is possible. In addition, quantum computers may utilize quantum processors capable of qubit-based parallel computation, and data processing efficiency can be improved through a hybrid structure with existing classical computers.
[0127] Meanwhile, the answer generation system (100) and the learning system according to the present invention may exist inside a server (hereinafter referred to as the server) built to perform a specific purpose (e.g., generating answers to queries and learning large language models), or they may exist as a separate system from said server. When the answer generation system (100) and the learning system exist inside the server, various services related to the present invention (e.g., providing answers to queries, generating learning data sets and learning large language models, etc.) may be provided through at least one component located inside the server, or through configuration modules that perform functions similar to said components. In this case, the application may provide various services related to the present invention on an electronic device (or user terminal (10)) on which the application is installed, through communication with the server.
[0128] Meanwhile, the answer generation system (100) according to the present invention may be configured to include a learning system (not shown). In this case, the learning system may provide various services related to the present invention (e.g., generation of a learning data set and learning of a large language model) through the communication unit (110), storage unit (120), control unit (130), and learning unit (140) of the answer generation system (100). As an example, the learning unit (140) of the answer generation system (100) may refer to the learning system.
[0129] In the present invention, the answer generation system (100) is described as including a learning system (or learning unit (140)) that performs learning on a large language model (131) to generate an answer to a query according to a step-by-step inference flow, but is not limited thereto. The learning system (or learning unit (140)) may be linked with an external server or a cloud computing environment to generate a learning data set for learning the large language model and perform a learning method for the large language model. At this time, the large language model (131) learned by the learning method of the large language model according to the present invention may be used in the answer generation system (100).
[0130] That is, the answer generation system (100) according to the present invention may include a learning system, or the learning system may exist as a separate and independent system from the answer generation system (100). For convenience of explanation, the answer generation system (100) is described below as including a learning system, and the learning system performs a learning method of a large language model through at least one of the communication unit (110), storage unit (120), control unit (130), and learning unit (140) of the answer generation system (100). Additionally, the answer generation system (100) and the learning system may be used in combination, and in this case, the answer generation system (100) may be described as performing the function of the learning system.
[0131] Meanwhile, the communication unit (110) according to the present invention may be connected via a wireless or wired network to a user terminal (10), an LLM server (150), a DB server, a cloud server, an external server, a central server, a device, and at least one network, etc., to receive or transmit overall data and information necessary for the operation of the answer generation system (100) according to the present invention.
[0132] The communication unit (110) can receive a user query (or, query) from the user terminal (10). Here, “receiving a user query (or, query)” may mean receiving an input signal corresponding to the user query based on the user query being input by the user through the input unit configuration provided in the user terminal (10).
[0133] In one embodiment, the communication unit (110) can transmit an answer to a query to a user terminal (10) in a real-time streaming manner. Specifically, in the process where a large language model (131) sequentially generates inference process data in token units, whenever inference data corresponding to each inference item is completed, the data can be transmitted to the user terminal (10) in real-time. Through this, the user can check the intermediate results of the inference process in real-time even before the entire answer generation is completed, and the responsiveness of the answer generation and the user experience can be improved.
[0134] Additionally, the communication unit (110) can transmit and receive query and answer data through a secure communication channel. For example, the security of the user's query content and answer data can be ensured by applying end-to-end encryption using TLS (Transport Layer Security) / SSL (Secure Sockets Layer) protocols. Furthermore, when used in sensitive domains such as medical, legal, and financial fields, security policies and data processing regulations suitable for the domain can be complied with.
[0135] Here, the user terminal (10) may include at least one of a mobile phone, a smartphone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, a wearable device (e.g., a smartwatch, a smart glass), and a head-mounted display (HMD).
[0136] In addition, the input section of the user terminal (10) in the present invention does not necessarily refer to a hardware means, but can be understood as a channel for receiving input from a user. Such an input section of the user terminal (10) may also be referred to as a user interface module. The input section of the user terminal (10) may include a touch screen, a computer mouse, a keyboard, a keypad, a touchpad, a trackball, a joystick, a voice recognition module, or other similar devices. However, the present invention does not limit the type of input section of the user terminal (10). Here, user input may include documents, text, images (or videos), voice, etc. In this case, the answer generation system (100) may further include a module for converting voice into text.
[0137] Meanwhile, the communication unit (110) may include at least one communication module capable of wireless communication and wired communication between the answer generation system (100) and at least one communication target. Additionally, the communication unit (110) may include a communication module that connects the answer generation system (100) to at least one network.
[0138] Meanwhile, the communication unit (110) can support various communication methods depending on the communication standard of the device being communicated. For example, the communication unit (110) may be configured to communicate with a communication target using at least one of the following technologies: WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth (Bluetooth™), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), and Wireless USB (Wireless Universal Serial Bus).
[0139] Next, a storage unit (120, or memory (not shown)) serves to store various data related to the present invention, and the storage unit (120) may be provided in the answer generation system (100) itself, or alternatively, at least part of the storage unit (120) may mean a database (DB) or memory. The storage unit (120) may include one or more non-transient computer-readable storage media that can be read or accessed by at least one processor. One or more computer-readable storage media may include volatile or non-volatile storage components such as optical, magnetic, organic, or other memory or disk storage devices. In some examples, the storage unit (120) may be implemented using a single physical device (e.g., one optical, magnetic, organic, or other memory or disk storage device), whereas in other examples, the storage unit (120) may be implemented using multiple physical devices.
[0140] The storage unit (120) may include computer-readable instructions and additional data. The storage unit (120) may include a storage necessary to perform at least some of the methods, scenarios, and techniques described in this specification, or at least some of the functions of the device and network. Furthermore, at least some of the storage unit (120) may be a cloud storage or a cloud server. That is, the storage unit (120) is sufficient as a space where information necessary for the operation of the answer generation system (100) according to the present invention is stored, and it can be understood that there are no restrictions on the physical space. Accordingly, the storage unit (120) and the database (not shown) may be used interchangeably without being separately distinguished below.
[0141] At least one data collected (or received) from various sources (e.g., a web corpus, a document corpus, a database (DB) website, an API, a server linked to the answer generation system (100), a central server, an external server, cloud storage, a user terminal (10), a large dataset, etc.) may be stored in the storage unit (120).
[0142] For example, a plurality of basic learning data collected from the various sources may be stored in the storage unit (120). Specifically, a plurality of basic learning data including at least one of query data, answer data, inference flow data, and inference scale data may be stored in the storage unit (120). At this time, domain data included in the learning query data may be stored in the storage unit (120). Furthermore, the storage unit (120) may include question-answer data including at least one query and correct answer data for each of the at least one query for generating a learning data set for training a large language model.
[0143] Additionally, a training data set for training a large language model may be stored in the storage unit (120). Specifically, at least one query and training inference process data corresponding to at least one query may be stored in the storage unit (120) as a training data set. Specifically, at least one of at least one query and training inference flow data, training inference data, and training answer data corresponding to at least one query may be stored in the storage unit (120) as the training data set.
[0144] In one embodiment, the storage unit (120) may include a vector database for efficient retrieval and updating of training data. Specifically, the vector database may store multiple query data and corresponding inference flow data converted into embedding vectors. Through this, the learning unit (140) or the control unit (130) can search for similar query data for a new query at high speed and quickly identify the corresponding reference training data set.
[0145] Additionally, incremental training data to support online learning or continual learning may be stored in the storage unit (120). Specifically, service log data including actual queries received from a user terminal (10) and user feedback thereon (e.g., answer quality evaluation, history of modifying inference items, etc.) may be processed into new training data and accumulated in the storage unit (120). Through continuous model updates using this accumulated data, the performance of the large language model (131) can be gradually improved during the service operation process.
[0146] Specifically, the storage unit (120) may store matching information in which reference query data, reference domain data, reference inference flow data, and reference inference scale data related to at least one query are matched. Here, the matching information may refer to data indicating the correspondence between the reference query data related to at least one query among multiple query data, the search result retrieved according to a preset similarity standard, and the reference domain data, reference inference flow data, and reference inference scale data corresponding to the selected reference query data.
[0147] Furthermore, a computer program including computer program instructions may be stored in the storage unit (120). Furthermore, a computer program including computer program instructions that control the operation of the answer generation system (100) or control the operation of the control unit (130) when loaded into the processor of the answer generation system (100) may be stored in the storage unit (120).
[0148] Specifically, the storage unit (120) may store data and commands necessary for the operation of the answer generation system (100) according to the present invention. For example, the storage unit (120) may store commands for the operation of a pre-trained large language model (131, or a reference large language model). The large language model (131) according to the present invention may include at least one artificial intelligence model trained to generate an answer to said query based on inference flow data related to the generation of an answer to a query and at least one inference data constituting the inference flow data. For example, the large language model (131) may include at least one artificial intelligence model trained to generate an answer to a query based on at least one large language model among EXAONE, T5 (Text-to-Text Transfer Transformer), BART (Bidirectional and Auto-Regressive Transformer), GPT (Generative Pre-trained Transformer), Gemini, DeepSeek, and LLaMA (Language Model for Many Applications), and based on inference flow data related to generating an answer to a query and at least one inference data constituting the inference flow data. At this time, the large language model (131) according to the present invention is not limited to the models described above and may further include large language models having the same function.
[0149] Additionally, data and commands necessary for the operation of the learning unit (140) may be stored in the storage unit (120). Specifically, the learning unit (140) may refer to a module capable of generating a learning data set for training a large language model to generate an answer to a query based on inference flow data related to the generation of an answer to a query, and training a large language model based on the generated learning data set.
[0150] Furthermore, the storage unit (120) may store data and commands necessary for the operation of the prompt generation unit. Specifically, the prompt generation unit may include at least one artificial intelligence model capable of generating a prompt that includes instructions requesting a large language model to perform at least one function. For example, the prompt generation unit may include at least one of EXAONE, T5 (Text-to-Text Transfer Transformer), BART (Bidirectional and Auto-Regressive Transformer), GPT (Generative Pre-trained Transformer), Gemini, DeepSeek, LLaMA (Language Model for Many Applications) based large language models, a rule-based template matching algorithm, a conditional prompting technique, a contextual embedding selection module, and a few-shot prompt generator. At this time, the artificial intelligence model included in the prompt generation unit according to the present invention is not limited to the described models and may further include artificial intelligence models that perform the same function.
[0151] Next, the control unit (130) can perform the role of controlling the overall operation of the answer generation system (100) and the learning system (not shown) related to the present invention. Specifically, the control unit (130) may include a large language model (131). At this time, although the present invention describes the control unit (130) as including a large language model (131), it is not limited thereto. If the large language model (131) exists on an external server (e.g., LLM server (150)), the control unit (130) may interact with the external server to call or control the functions of the pre-trained large language model (131).
[0152] For example, if at least one of the large language model (131) according to the present invention and the reference large language model used for learning the large language model exists in the LLM server (150), the control unit (130) can call or control the functions of the large language model (131) and the reference large language model in conjunction with the LLM server (150). At this time, although the LLM server (150) is described as existing separately from the answer generation system (100) in the present invention, it is not limited thereto, and the answer generation system (100) may be configured to include the LLM server (150). That is, the answer generation system (100) and the LLM server (150) according to the present invention may exist separately, or the LLM server (150) may be included in the answer generation system (100). For convenience of explanation, the LLM server (150) and the large language model (131, or reference large language model) may be used interchangeably, and the use of the large language model (131, or reference large language model) by the control unit (130) can be understood as using at least one large language model included in the LLM server (150).
[0153] For example, the LLM server (150) of the present invention may include at least one large language model based on at least one of EXAONE, T5 (Text-to-Text Transfer Transformer), BART (Bidirectional and Auto-Regressive Transformer), GPT (Generative Pre-trained Transformer), Gemini, DeepSeek, and LLaMA (Language Model for Many Applications). At this time, the large language model included in the LLM server (150) according to the present invention is not limited to the models described above, and the LLM server (150) according to the present invention may include an artificial intelligence model that performs the same function as the large language model.
[0154] The control unit (130) can process signals, data, information, etc. that are input or output through the components of the answer generation system (100) described above, or perform a series of data processing to provide or process appropriate information and functions to the user. The control unit (130) can be physically implemented by the processor described above.
[0155] As illustrated in FIG. 5a, the control unit (130) can receive a query (510) from a user terminal (10). Here, the query (or user query) may refer to request information entered by the user to obtain specific information or solve a problem, and the query may include multimodal input data composed of text data, image data, voice data, video data, or a combination thereof.
[0156] Furthermore, the control unit (130) can generate inference flow data (520) in response to a query. At this time, the method by which the control unit (130) generates inference flow data (520) in response to a query can be very diverse. For example, the control unit (130) can control a large language model to generate a thought structure for generating an answer to a query based on a query, and accordingly, the large language model can generate inference flow data including multiple inference items for generating an answer to a query.
[0157] As another example, the control unit (130) may use a prompt template that is pre-stored in the storage unit (120) to generate inference flow data. For example, the control unit (130) may provide a prompt template defining a thought structure as input to a large language model, thereby enabling inference flow data to be generated based on the prompt template.
[0158] As another example, the control unit (130) may generate inference flow data using a memory context that includes at least one of inference flow data, inference items, inference data, and inference process data related to a previously entered query. For example, the control unit (130) may provide the memory context stored in the storage unit (120) as input to a large language model so that inference flow data based on past information related to the query is generated.
[0159] Additionally, the control unit (130) may construct reasoning flow data based on reasoning tokens in token units generated during the processing of a large language model. For example, the control unit (130) may analyze the token information generated during the process of generating reasoning flow data to generate reasoning flow data containing reasoning items according to a stepwise reasoning structure.
[0160] Accordingly, the inference flow data according to the present invention may be generated directly by a large language model, or may be generated based on prompt templates, memory contexts, or token-based intermediate information, and may be generated in various ways without being limited to these methods.
[0161] Specifically, the control unit (130) can generate inference flow data (520) consisting of at least one inference item related to generating an answer to a query by using a pre-trained large language model (131). Furthermore, the control unit (130) can obtain inference data corresponding to at least one inference item included in the inference flow data (520). At this time, the method by which the control unit (130) obtains inference data corresponding to at least one inference item can be very diverse. For example, the control unit (130) can generate inference data corresponding to at least one inference item included in the inference flow data (520) using the large language model (131) and obtain the generated inference data. In addition, the control unit (130) can obtain inference data corresponding to at least one inference item included in the inference flow data (520) from an external server through a retrieve process. For example, the control unit (130) can generate a query or a search query corresponding to the inference item. Furthermore, the control unit (130) can use a search query to search for information corresponding to at least one inference item included in the inference flow data (520) from at least one of an external server, a database, a document repository, and a knowledge base, and can obtain the searched information as inference data.
[0162] Furthermore, the control unit (130) may combine a generation process by a large language model and a search process from an external server to obtain inference data corresponding to at least one inference item included in the inference flow data (520). For example, the control unit (130) may provide information searched from an external server as input to the large language model so that inference data based on the searched information is generated.
[0163] Accordingly, the inference data according to the present invention may be generated and obtained by a large language model, or obtained through a search from an external server or information repository, or obtained in various ways combining such generation and search processes. Specifically, the control unit (130) can generate inference data for each of at least one inference item constituting the inference flow data using a pre-trained large language model (131). Specifically, the large language model (131) can sequentially generate inference data according to a pre-set order for at least one inference item constituting the inference flow data.
[0164] The above inference flow data may be generated to include multiple inference items based on the type of query, problem difficulty, or domain information. For example, inference items may include problem interpretation, setting of approach strategies, calculation, verification, etc.
[0165] As a more specific example, inference flow data may be generated to include multiple inference items based on the semantic analysis results of a query, problem type, or domain information. The inference items may include at least one of problem interpretation, setting of an approach strategy, intermediate calculation, and result verification, and the number or order of the inference items may be dynamically determined according to the characteristics of the query.
[0166] Furthermore, the control unit (130) can provide inference data and a query as inputs to a pre-trained large language model (131) and obtain an answer to the query from the large language model (131). For example, the control unit (130) can control the large language model (131) to sequentially combine at least one inference data generated to generate inference process data. Furthermore, the control unit (130) can obtain at least a portion of the inference process data, including answer data to the query generated based on at least one inference data, as an answer to the query. Thus, in the present invention, “inference process data” refers to a data structure in which a plurality of inference data are sequentially combined, and for example, answer data to the query may be additionally included in the inference process data.
[0167] The above inference process data may be a data set containing multiple intermediate inference results generated during the query resolution process, and the data set may be formed to maintain relationships between inference steps.
[0168] The control unit (130) can provide an answer to a query to the user terminal. Specifically, the control unit (130) can provide an answer to a query to the user terminal through at least one service page (1000). At this time, the service page (1000) is a user interface displayed on the output unit (or display unit) of the user terminal (10), and may mean one or more screen configurations for providing an answer to a query and information related to the generation of said answer. For example, the control unit (130) can provide an answer to a query received through the service page (1000) to the user terminal (10) by displaying it in a form including text, a table, a chart, an image, video, audio, or a combination thereof. Furthermore, the control unit (130) can receive user input received from the user terminal (10) through the service page. User input according to the present invention may be performed in at least one of pointing input, over input, positioning input (Ex: hovering), touch, tap, double tap, long press, click, swipe, drag, drag and drop, pinch in, pinch out, and rotate.
[0169] At this time, the user input according to the present invention is not limited to the examples described above, and may include all forms of input methods that allow the user's intention (e.g., selection, etc.) to be reflected with respect to at least one of the user interface, graphic object, and icon provided on the service page.
[0170] Additionally, the control unit (130) can use the learning unit (140) to train the large language model (131) to be trained so that the large language model (131) according to the present invention generates an answer to a query according to a step-by-step inference flow. Specifically, the control unit (130) can use the learning unit (140) to generate a training data set used for training the large language model (131). At this time, the operation performed by the learning unit (140) according to the present invention can also be described as being performed by the control unit (130).
[0171] As illustrated in FIG. 5b, the learning unit (140) can specify a reference learning data set to be used to generate a learning data set by using at least some of the multiple basic learning data (540) already stored in the storage unit (120). Specifically, the learning unit (140) can specify at least one query to generate a learning data set. For example, the learning unit (140) can specify at least one query (550) to be included in the learning data set by selecting at least some of the multiple query data collected from various sources. At this time, correct answer data may be matched to each of the multiple query data.
[0172] Specifically, the learning unit (140) can identify (or search, determine, etc.) reference query data (561), reference inference flow data (562), and reference inference scale data (563) related to at least one query (550). For example, the learning unit (140) can identify candidate queries (or at least one candidate query data) corresponding to at least one query from a plurality of basic learning data according to a preset similarity standard. Here, the preset similarity standard can be set in a wide variety of ways. At this time, there may be multiple candidate query data corresponding to at least one query according to the present invention.
[0173] For example, the learning unit (140) can calculate domain similarity between domain data corresponding to at least one query and candidate domain data corresponding to each of the candidate query data. Furthermore, based on the domain similarity, the learning unit (140) can specify reference query data among the candidate query data and specify a reference learning data set including at least one of reference inference flow data and reference inference scale data corresponding to the reference query data.
[0174] As another example, the learning unit (140) can identify candidate inference scale data corresponding to each candidate query data and calculate the inference scale similarity between the candidate inference scale data.
[0175] In the present invention, “inference scale data” may refer to metadata indicating the length or complexity of an inference process. For example, the inference scale data may include at least one of the number of inference steps, the number of generated inference tokens, or the length of the inference process data.
[0176] Furthermore, the learning unit (140) may specify a reference query data among candidate query data based on the similarity of the inference scale, and specify a reference learning data set that includes at least one of reference inference flow data and reference inference scale data corresponding to the reference query data. At this time, the reference query data, reference inference flow data, and reference inference scale data included in the reference learning data set may be matched and stored in the storage unit (120).
[0177] Furthermore, the learning unit (140) can generate learning inference flow data (570) related to generating an answer to a query (550) using a reference learning data set. Specifically, the learning unit (140) can generate learning inference flow data (570) related to generating an answer to a query (550) using a reference large language model (160). The reference large language model (160) may refer to a large language model capable of generating data necessary to generate a learning data set corresponding to at least one query.
[0178] In the present invention, the reference large language model is a large language model distinct from the large language model (131) that is the subject of learning, and may be a large language model used to generate a learning data set. The reference large language model (160) may be included in at least one of the answer generation system (100), the control unit (130), the learning unit (140), and the LLM server (150). In the present invention, the answer generation system (100) or the learning system may be implemented to generate a learning data set using the reference large language model provided from an external server via a network. That is, the storage location or operation method of the reference large language model (160) according to the present invention is not limited to specific examples.
[0179] Furthermore, the learning unit (140) can generate learning inference data (580) for each of at least one inference item constituting the learning inference flow data (570). Specifically, the learning unit (140) can generate learning inference data (580) corresponding to each of at least one inference item constituting the learning inference flow data (570) by using a reference large language model (160). Furthermore, the learning unit (140) can generate learning inference process data (590) composed of learning inference data (580) corresponding to each of at least one inference item and learning answer data for a query by using a reference large language model (160).
[0180] Furthermore, the learning unit (140) can generate a learning data set for learning a large language model based on the learning inference data. Specifically, the learning unit (140) can generate a learning data set including at least one query (550) and learning inference process data (590) corresponding to the query (550).
[0181] The learning unit (140) can train a large language model (131) to be trained using a learning data set. The training method for a large language model according to the present invention may be very diverse. In this specification, any training method capable of training a large language model is not limited to any specific type or method. For example, the learning unit (140) can train (or fine-tune) the large language model (131) by updating the parameters of the large language model (131) based on the learning data set so that the large language model (131) generates an answer to a query according to the inference flow. Here, the parameters of the large language model (131) are at least among weight values, bias values, embedding parameters, vision encoder parameters, text encoder parameters, patch embedding parameters, the query / key / value (Q / K / V) matrix of the multi-head attention module, the linear transformation coefficients of the feedforward network (FFN), the layer normalization coefficients, the weights of the cross-attention combining module, the output matrix for calculating the token generation logit, the scale coefficients and offset coefficients after the activation function, the weight matrix of the multimodal combining module combining image features and text features, the position embedding, the segment embedding, the representation vectors of the class token and the image patch token, the learning rate of the vision language model, the dropout rate and the regularization coefficient. It can mean all learnable parameters including one.At this time, the parameters of the large language model (131) being updated are not limited to the examples described above, and may include various additional parameters for training (or optimizing, fine-tuning) the large language model.
[0182] At this time, the learning method according to the present invention is not limited to the examples described above and may vary greatly depending on the structure of the large language model (131), learning conditions, objective function setting, or the configuration of the learning data. As an example, the large language model (131) may be learned (or fine-tuned) in at least one of supervised fine-tuning, preference-based tuning, RLHF (Reinforcement Learning from Human Feedback) based fine-tuning, LoRA (Low-Rank Adaptation) based fine-tuning, adapter-based fine-tuning, prompt-tuning, prefix-tuning, and multimodal alignment fine-tuning.
[0183] Meanwhile, the answer generation system (100) and the learning system (not shown) may include at least one processor, and such processor may include one or more general-purpose processors or one or more special-purpose processors (e.g., digital signal processor, tensor processing unit (TPU), graphics processing unit (GPU), neural network processing unit (NPU), application integrated circuit, application semiconductor (ASIC), etc.). At least one processor may be configured to execute instructions, computer-readable instructions, or other instructions described herein that are stored (or included) in the storage unit (120). Such an answer generation system (100) and the learning system may perform data processing described below in cooperation with memory and at least one processor. The processor may be electrically connected to memory and may perform a series of operations and data processing using data and information stored in memory. Here, “memory” may be a component of the storage unit (120), and “processor” may be used interchangeably with the control unit (130).
[0184] In the above description, the answer generation system (100) and the learning system of the present invention have been described, and this can implement the answer generation method and the learning method of a large language model described below.
[0185] Hereinafter, a method for learning a large language model according to the present invention will be described in more detail with reference to FIGS. 7a to 7d and FIGS. 8a to 8c. FIGS. 7a to 7d and FIGS. 8a to 8c are conceptual diagrams for explaining a method for learning a large language model according to the present invention.
[0186] The training method for a large language model according to the present invention is broadly composed of (1) a step of specifying a reference training data set, (2) a step of generating training inference flow data, (3) a step of generating training inference data, (4) a step of configuring and filtering a training data set, and (5) a large language model training step. These steps may be performed sequentially in a pipeline form, but some steps may be performed in parallel or repeatedly. Each step is described in detail below with reference to the drawings.
[0187] As previously explained, the answer generation system (100) according to the present invention may be configured to include a learning system. In this case, the learning system may provide various services related to the present invention (e.g., generation of a learning data set and training of a large language model) through the communication unit (110), storage unit (120), control unit (130), and learning unit (140) of the answer generation system (100). As an example, the learning unit (140) of the answer generation system (100) may refer to the learning system.
[0188] In the present invention, the answer generation system (100) is described as including a learning system (or learning unit (140)) that performs learning on a large language model (131) to generate an answer to a query according to a step-by-step inference flow, but is not limited thereto. The learning system (or learning unit (140)) may be linked with an external server or a cloud computing environment to generate a learning data set for learning the large language model and to perform learning of the large language model. At this time, the large language model (131) learned by the large language model learning method according to the present invention may be used in the answer generation system (100).
[0189] That is, the answer generation system (100) according to the present invention may include a learning system, or the learning system may exist as a separate and independent system from the answer generation system (100). For convenience of explanation, the answer generation system (100) is described below as including a learning system, and the learning system performs a learning method of a large language model through at least one of the communication unit (110), storage unit (120), control unit (130), and learning unit (140) of the answer generation system (100). Additionally, the answer generation system (100) and the learning system may be used in combination, and in this case, the answer generation system (100) may be described as performing the function of the learning system.
[0190] Furthermore, the operation performed by the learning unit (140) in the present invention can also be understood as an operation performed by the control unit (130), and for convenience of explanation, each step of the learning method of a large language model can be understood as being performed by at least one of the learning unit (140) and the control unit (130). Below, the process of generating (or constructing) a learning data set used for learning a large language model according to the present invention will be described.
[0191] In the present invention, a step of specifying a reference learning data set corresponding to at least one query may be performed.
[0192] As illustrated in FIG. 7a, the learning unit (140) can identify at least one query (550) that is the target for generating a learning data set. Specifically, the learning unit (140) can identify at least one query and an answer corresponding to said at least one query from at least one data collected (or received) from various sources (e.g., a web corpus, a document corpus, a database (DB, 200), a website, an API, a server linked to an answer generation system (100), a central server, an external server, cloud storage, a user terminal (10), a large-scale data set, etc.).
[0193] Furthermore, the learning unit (140) can identify (or search for) a reference learning data set (560) from a plurality of basic learning data stored in a database (200 or storage unit (120)) using at least one query. At this time, the plurality of basic learning data may be configured by matching at least one of a plurality of query data, inference flow data for each of the plurality of query data, inference scale data, and answer data. As previously explained, the inference flow data may refer to data configured according to a preset order of at least one inference item performed to generate an answer to a query.
[0194] In this case, an inference item may refer to information on individual inference units performed to derive an answer to a query. Additionally, inference scale data may include information indicating the number of inference tokens used in the inference process for the query, the number of inference steps, and the length of the inference process.
[0195] Specifically, the learning unit (140) can identify candidate query data corresponding to at least one query from a plurality of basic learning data according to a preset similarity standard. Here, a candidate query corresponding to at least one query may refer to a query extracted (or searched) from a plurality of basic learning data based on similarity with at least one query. At this time, there may be multiple candidate query data corresponding to at least one query according to the present invention.
[0196] The method for specifying candidate query data corresponding to at least one query according to the present invention can be very diverse, and the present specification is not limited to any type or method as long as it is a method capable of specifying candidate query data corresponding to at least one query. Specifically, the pre-set similarity criteria according to the present invention can be set in very diverse ways. For example, the similarity criteria can be set for text-to-text similarity, semantic similarity, embedding vector-to-embedding similarity, keyword-based similarity, or a combination thereof. As an example, the similarity criteria can be set in various ways, such as a method of selecting the top K queries in order of highest similarity value, a method of selecting queries whose similarity value is greater than or equal to a pre-set threshold, or a method of selecting queries included within a certain range based on similarity values. Furthermore, the similarity criteria can be set to specify candidate query data based on the ranking of similarity values, a threshold, or a combination thereof, but is not limited thereto.
[0197] Additionally, the similarity according to the present invention may be calculated using cosine similarity, inner product similarity, Euclidean distance, BM25-based similarity, or other information retrieval-based similarity calculation methods, but is not limited thereto. For example, the learning unit (140) may specify the candidate query data based on a Retrieval-Augmented Generation (RAG) method. Specifically, the learning unit (140) may perform a search for query data included in a plurality of basic learning data based on at least one query that is the target for creating a learning data set, and may specify a query that satisfies a pre-set similarity standard with the at least one query as a candidate query. As an example, the learning unit (140) may convert at least one query that is the target for creating a learning data set into an embedding vector, calculate the similarity between the embedding vector and the embedding vector corresponding to each of the query data included in the plurality of basic learning data, and specify the candidate query data in order of highest similarity.
[0198] At this time, the learning unit (140) may specify a reference query data among candidate query data by using domain data corresponding to at least one query that is the target for generating the learning data set, in order to generate a learning data set corresponding to at least one query suitable for the domain (or topic, problem area, knowledge area, etc.) of the query. Specifically, the domain data may include information indicating the problem type, knowledge area, or task type to which the query belongs, and may include domains such as, for example, mathematics, physics, coding, logical reasoning, or common-sense reasoning, but is not limited thereto.
[0199] The learning unit (140) can calculate domain similarity between domain data corresponding to at least one query and candidate domain data corresponding to candidate query data. The method for calculating domain similarity according to the present invention may apply the method for calculating similarity.
[0200] As a specific embodiment, it is assumed that the domain data is a 'math' domain. The learning unit (140) can calculate an embedding vector for a query belonging to the 'math' domain (e.g., solving equations, calculating probability, geometry problems, etc.) and calculate the cosine similarity with each embedding vector of candidate query data included in a plurality of basic learning data. For example, regarding the domain data of a query 'solve a system of equations', the domain similarity with each of the candidate queries included in the basic learning data, such as 'solving quadratic equations', 'matrix operations', and 'calculus calculations', can be calculated, and the top K reference query data can be identified in order of high similarity. At this time, the value of K can be set differently for each domain; for example, K can be set to 5 for the math domain and K to 3 for the law domain, but is not limited thereto.
[0201] In another embodiment, when the domain data is a 'coding' domain, the learning unit (140) can calculate domain similarity by additionally considering sub-domain attributes such as programming language type, algorithm type, and problem difficulty. For example, for a sorting algorithm problem written in Python, domain similarity can be calculated by assigning weights to the same programming language and algorithm type, thereby more precisely identifying similar reference query data.
[0202] Furthermore, the learning unit (140) may specify a reference query data among candidate query data based on domain similarity, and specify a reference learning data set including at least one of reference inference flow data and reference inference scale data corresponding to the reference query data. For example, the learning unit (140) may specify a reference query data in which the domain similarity between at least one query and candidate query data that is the target for generating the learning data set satisfies a pre-set domain similarity standard. Here, the pre-set domain similarity standard according to the present invention can be set in a wide variety of ways. For example, the domain similarity standard can be set in various ways, such as a method of selecting the top K query data in order of highest domain similarity value, a method of selecting query data in which the domain similarity value is greater than or equal to a pre-set threshold, or a method of selecting queries included within a certain range based on the domain similarity value. In addition, the domain similarity standard can be set to specify at least one reference query data based on the ranking of similarity values, a threshold, or a combination thereof, but is not limited thereto.
[0203] Additionally, the learning unit (140) may calculate the inference scale similarity between candidate query data and specify a reference query data among the candidate query data based on the inference scale similarity so that the length of the inference process data or the number of inference steps for at least one query that is the target of generating a learning data set corresponds appropriately to the complexity of the query. Specifically, the inference scale data may include information indicating the number of inference tokens used in the inference process for the query, the number of inference steps, and the length of the inference process. For example, in the case of a relatively simple query, reference query data corresponding to an inference scale relatively smaller than that of a complex query may be referenced, and in the case of a relatively complex query, reference query corresponding to an inference scale relatively larger than that of a simple query may be referenced.
[0204] In this way, by considering the inference scale similarity between candidate query data, the learning unit (140) can identify query data having a similar level of inference scale data as reference query data. Specifically, the learning unit (140) can identify candidate inference scale data corresponding to candidate query data and calculate the inference scale similarity between candidate inference scale data. The method for calculating the inference scale similarity according to the present invention may apply the method for calculating similarity.
[0205] Furthermore, the learning unit (140) may specify a reference query data among candidate query data based on inference scale similarity, and specify a reference learning data set including at least one of reference inference flow data and reference inference scale data corresponding to the reference query data. For example, the learning unit (140) may specify a reference query data in which the inference scale data of each candidate query data satisfies a preset inference scale similarity standard. Here, the preset inference scale similarity standard according to the present invention may be set in a wide variety of ways. For example, the inference scale similarity standard may be set based on at least one of whether the difference between the inference scale data corresponding to the candidate query data is included within a preset range, whether the ratio between the inference scale data is included within a preset range, and whether the similarity value between the inference scale data is greater than or equal to a preset threshold. Additionally, inference scale similarity can be calculated based on the difference, ratio, and normalized similarity value between the number of inference tokens corresponding to candidate query data; for example, it can be calculated using a similarity value derived based on the ratio of the smaller and larger values among two inference scale data, but is not limited thereto.
[0206] As illustrated in FIG. 7b, the learning unit (140) can specify a reference learning data set (560) corresponding to at least one query by using reference inference flow data (562) and reference inference scale data (563) matched with at least one specified reference query data (561). Furthermore, the learning unit (140) can store a reference learning data set including at least one of reference query data (561), reference domain data (564), reference inference flow data (562), and reference inference scale data (563) in a database (200, or storage unit (120)). For example, the learning unit (140) can organize the reference query data (561), reference domain data (564), reference inference flow data (562), and reference inference scale data (563) into a single record associated with each other and store it in the database (200) or storage unit (120).
[0207] As an example, the reference training data set may be stored in a data table structure in a form where each item is composed of corresponding fields, for example, reference query data (561), reference domain data (564), reference inference flow data (562), and reference inference scale data (563) may be stored as respective columns or attributes.
[0208] However, it is not limited thereto, and the reference training dataset may be stored in various forms of data storage structures, including key-value structures, document structures, graph structures, or other data structures. Additionally, the reference training dataset may be stored in various ways, such as in the form of a single data object, in the form of association relationships between multiple data objects, or in a form distributed and stored across multiple data files.
[0209] Next, in the present invention, a step of generating learning inference flow data (or learning thought structure data) related to the generation of an answer to the above query can be performed using a reference learning data set.
[0210] The learning unit (140) can generate learning inference flow data related to the generation of an answer to at least one query using a reference large language model (160). As previously described, the reference large language model (160) may refer to a large language model trained to generate data necessary to generate a learning data set corresponding to at least one query.
[0211] As illustrated in FIG. 7c, the learning unit (140) may provide (or process) the reference query data (561), reference inference flow data (562), reference inference scale data (563), and the at least one query (550) as input to a pre-trained reference large language model (160). At this time, the learning unit (140) may generate a first prompt (710) consisting of at least one instruction requesting the generation of learning inference flow data (570) corresponding to at least one query (550) based on the reference learning data set (560).
[0212] The method for generating a prompt according to the present invention may vary widely, and the control unit (130) according to the present invention may further include a prompt generating unit for generating a prompt. For example, the prompt generating unit may generate a first prompt (710) consisting of at least one instruction requesting the generation of learning inference flow data (570) corresponding to at least one query (550) based on a reference learning data set (560) using at least one artificial intelligence model.
[0213] Furthermore, the learning unit (140) may provide (or process) the first prompt (710) as input to the reference large language model (160) along with the reference learning data set (560) and at least one query (550). The reference large language model (160) may generate learning inference flow data (570) related to generating an answer to at least one query (550) based on the reference learning data set (560). At this time, the learning inference flow data (570) may be configured to include at least one inference item (571, 572) according to a preset order according to the inference flow for generating an answer to at least one query (550).
[0214] Specifically, the learning inference flow data (570) may include multiple inference items (571, 572) according to a preset order based on the logical sequence of the inference flow performed to generate an answer to at least one query (550). For example, the learning inference flow data (570) may be configured such that inference items corresponding to problem understanding, analysis for problem solving, execution of the solving process, and result verification, respectively, are arranged sequentially according to the preset order. In this way, the learning unit (140) can generate the learning inference flow data (570) in a structured form in which multiple inference items are arranged according to the logical sequence for generating an answer to the query.
[0215] Meanwhile, the learning unit (140) can predict the number of at least one inference item by using at least one of reference inference flow data and reference inference scale data. The number of inference items can be determined based on the length of the query, the type of problem, or a learned difficulty prediction model.
[0216] At this time, the learning unit (140) can determine or predict the number of inference items according to the characteristics of the query. For example, the number of necessary inference steps can be calculated based on the length of the query, the semantic complexity of the query, or the domain information to which the query belongs. In addition, the number of inference items can be predicted using a difficulty prediction model or an inference scale prediction model learned from past query and inference process data.
[0217] Furthermore, the learning unit (140) can control the operation of the reference large language model (160) to generate learning inference flow data based on the predicted number of inference items. For example, the learning unit (140) can configure the first prompt (710) to include at least one instruction requesting to estimate the number of inference items required to generate an answer to at least one query using at least one of the reference inference flow data and reference inference scale data included in the reference learning data set.
[0218] As an example, the learning unit (140) may configure a first prompt (710) so that the reference large language model (160) calculates the number of inference items required to generate an answer to the query based on at least one of the number of inference items included in the reference inference flow data (562) and the reference inference scale data (563). The learning unit (140) may control the reference large language model (160) to generate learning inference flow data that includes an inference item corresponding to the number of inference items based on the number of inference items calculated by the reference large language model (160) according to the first prompt (710). Furthermore, the reference large language model (160) may generate learning inference flow data that includes at least one inference item corresponding to the number of calculated inference items.
[0219] Next, in the present invention, a step of acquiring (or generating) learning inference data (or learning basis data) corresponding to at least one inference item constituting learning inference flow data (or learning thought structure data) may be performed.
[0220] As illustrated in FIG. 7d, the learning unit (140) can generate learning inference data (580) for each of at least one inference item constituting the learning inference flow data (570) using a reference large language model (160). Specifically, the learning unit (140) can provide (or process) the learning inference flow data (570) as input to the reference large language model (160). Furthermore, the learning unit (140) can control the reference large language model (160) to generate learning inference data corresponding to each of the at least one inference item constituting the learning inference flow data (570).
[0221] For example, the learning unit (140) may generate a prompt requesting the generation of learning inference data using at least one query, pre-generated learning inference data, a first inference item, and a second inference item. Here, the first inference item may refer to an inference item whose pre-set order according to the inference flow precedes the second inference item. Specifically, the first inference item may be an inference item that immediately precedes the second inference item, but is not limited thereto and may be interpreted as a concept including at least one of a plurality of inference items that precede the second inference item. For example, the first inference item may be an inference item corresponding to the step immediately preceding the second inference item, or it may refer to a set of inference items that includes all or part of a plurality of inference items that precede the second inference item.
[0222] Additionally, according to an embodiment of the present invention, the first inference item may mean a single-stage inference item preceding the second inference item, or may mean a cumulative inference item including multiple-stage inference items preceding the second inference item, and such configuration is not limited to a specific form.
[0223] Furthermore, the learning unit (140) may provide a prompt along with the learning inference flow data as input to a pre-trained reference large language model. The reference large language model (160) may generate learning inference data by referring to a second inference item to generate first learning inference data corresponding to a first inference item based on the prompt. For example, the learning unit (140) may generate a second prompt (720) consisting of at least one instruction requesting the generation of learning inference data corresponding to each of at least one inference item constituting the learning inference flow data (570).
[0224] Specifically, the learning unit (140) can generate a second prompt (720) consisting of at least one instruction requesting the sequential generation of learning inference data (580) according to a preset order for at least one inference item constituting the learning inference flow data (570) by using a prompt generation unit. As an example, the learning unit (140) can request the reference large language model (160) to prioritize the generation of first learning inference data (581) corresponding to the first inference item (571) based on the second prompt (720).
[0225] Meanwhile, the learning unit (140) can control the reference large language model (160) to generate inference data corresponding to the next set order of inference items by using at least one learning inference data corresponding to the inference item that is ahead in the preset order. The reference large language model (160) can sequentially generate learning inference data corresponding to the next order of inference items by referring to at least one learning inference data corresponding to the inference item that is ahead in the preset order. For example, the second prompt (720) can be configured to generate first learning inference data (581) corresponding to the first inference item (571), and to generate second learning inference data (582) by referring to the first learning inference data (581).
[0226] Furthermore, the standard large language model (160) can generate first learning inference data (581) corresponding to the first inference item (571) based on the second prompt (720), and generate second learning inference data (582) by referring to the first learning inference data (581), and can be configured so that inference data for the next stage is generated using inference data corresponding to the inference item that precedes the inference item in a preset order for the specific inference item.
[0227] At this time, for the reference large language model (160) to generate learning inference data (580), the learning inference data corresponding to the inference item with a preset order that is referenced is not limited to the learning inference data generated immediately prior, and may include at least some or all of the learning inference data (580) corresponding to a plurality of inference items with a preset order that is prior.
[0228] For example, the reference large language model (160) may refer only to the training inference data generated immediately prior to the next inference item in the process of generating training inference data corresponding to the next inference item, may refer together to multiple training inference data generated earlier in time, or may be configured to refer only to some training inference data selected according to a pre-set standard. As such, in the present invention, the range, number, or selection method of the training inference data referenced by the reference large language model (160) can be set in a wide variety of ways and is not limited to a specific reference method.
[0229] Furthermore, the learning unit (140) may request the reference large language model (160) to generate second learning inference data (582) corresponding to the second inference item (572) by including the first learning inference data (581) generated from the reference large language model (160) as an input condition. That is, the second prompt (720) may be configured to instruct the inference corresponding to the second inference item (572) to be performed, while simultaneously allowing the inference of the next step to be performed by referring to the first learning inference data (581) generated in the previous step. Furthermore, the reference large language model (160) may sequentially generate learning inference data (580) according to a pre-set order of at least one inference item constituting the learning inference flow data (570).
[0230] In the present invention, the step of generating learning inference flow data (570) and the step of generating learning inference data (580) are described as being performed individually, but are not limited thereto. That is, in one embodiment of the present invention, the step of generating learning inference flow data (570) and the step of generating learning inference data (580) are described as being performed sequentially, but in other embodiments, the step of generating learning inference flow data (570) and the step of generating learning inference data (580) may be performed simultaneously, or some steps may be performed in an integrated form.
[0231] For example, the reference large language model (160) may be configured to perform the generation of learning inference flow data (570) and the generation of learning inference data (580) in a single process, and in this case, learning inference data corresponding to at least one inference item (571, 572) constituting the learning inference flow data (570) may be generated simultaneously or sequentially.
[0232] Additionally, although the present invention has been described as having a first prompt (710) and a second prompt (720) composed of separate prompts, it is not limited thereto, and in other embodiments, the first prompt (710) and the second prompt (720) may be composed of a single integrated prompt and provided to a reference large language model (160). In this case, the integrated prompt may be configured to include together an instruction requesting the generation of learning inference flow data (570) and an instruction requesting the generation of learning inference data (580) corresponding to each inference item.
[0233] Furthermore, the processing method, the order of execution of steps, the prompt configuration method, or the data generation method for generating learning inference flow data (570) and learning inference data (580) according to the present invention may be modified or combined in various forms within the scope of not departing from the technical spirit of the present invention, and is not limited to the specific implementation method described in this specification.
[0234] Meanwhile, the learning unit (140) can control the reference large language model (160) to generate learning inference flow data and learning inference data based on at least one of reference domain data and reference inference scale data. Specifically, the reference large language model (160) can generate learning inference flow data and learning inference data based on reference domain data and reference inference scale data corresponding to at least one query (550) that is the target of generating the learning data set.
[0235] For example, as illustrated in FIG. 8a, the learning unit (140) may provide a first query (550a) and a first reference learning data set (560a) as inputs to a reference large language model (160). In this case, the first reference learning data set (560a) may include at least one of a first reference domain data (e.g., simple calculation) and a first reference inference scale data (e.g., token length 150).
[0236] A reference large language model (160) can generate a first learning inference flow data (570a) based on at least one of a first reference domain data and a first reference inference scale data corresponding to a first query (550a). Furthermore, the reference large language model (160) can sequentially generate first learning inference data (581a, 582a) corresponding to each of at least one inference item constituting the first learning inference flow data (570a).
[0237] As another example, as illustrated in FIG. 8b, the learning unit (140) may provide a second query (550b) and a second reference learning data set (560b) as inputs to a reference large language model (160). In this case, the second reference learning data set (560b) may include at least one of a second reference domain data (Ex: law) and a second reference inference scale data (Ex: token length 500).
[0238] A reference large language model (160) can generate second learning inference flow data (570b) based on at least one of second reference domain data and second reference inference scale data corresponding to a second query (550b). Furthermore, the reference large language model (160) can sequentially generate second learning inference data (581b to 585b) corresponding to each of at least one inference item constituting the second learning inference flow data (570b).
[0239] In this way, the learning unit (140) can control the reference large language model to generate inference flow data and inference data in a form suitable for said at least one query based on the domain to which said at least one query, which is the target of generating the learning data set, belongs and the scale of inference required for generating the answer. As an example, the learning unit (140) may generate inference flow and inference data having a relatively small scale of inference in the case of a query requiring simple inference, such as a relatively simple calculation problem, and may generate inference flow and inference data having a relatively large scale of inference in the case of a query requiring complex inference, such as legal analysis.
[0240] Accordingly, the learning unit (140) according to the present invention can generate learning inference flow data and learning inference data suitable for the characteristics of the query by using reference domain data and reference inference scale data corresponding to at least one query that is the target of generating a learning data set.
[0241] Meanwhile, the learning unit (140) can generate learning inference data corresponding to each of at least one inference strategy data included in the learning inference flow data by reflecting at least one inference strategy data corresponding to each of at least one inference item included in the learning inference flow data. Here, the inference strategy data may refer to data representing at least one of a problem-solving method, an analysis procedure, and an inference approach applied to generate an answer to a query.
[0242] Specifically, the learning unit (140) can generate inference strategy data including at least one of necessary mathematical tools, algorithm selection, calculation method, and logical judgment criteria. For example, the learning unit (140) can generate inference strategy data corresponding to each of at least one inference item constituting the learning inference flow data using a reference large language model (160). The learning unit (140) can generate a prompt (or a third prompt) including at least one instruction requesting the generation of inference strategy data corresponding to each of at least one inference item constituting the learning inference flow data using a prompt generation unit.
[0243] Furthermore, the learning unit (140) may provide (or process) a prompt containing at least one instruction requesting the generation of inference strategy data as input to the reference large language model (160). The reference large language model (160) may generate inference strategy data corresponding to each of at least one inference item constituting the learning inference flow data.
[0244] The learning unit (140) can generate learning inference data corresponding to each of at least one inference item constituting the learning inference flow data by using at least one inference strategy data generated by the reference large language model (160). Specifically, the learning unit (140) can configure a prompt (810 or a second prompt) so that the inference strategy data is reflected in the learning inference data generation process, and can provide (or process) the second prompt as an input to the reference large language model (160). Furthermore, the reference large language model (160) can generate learning inference data corresponding to each of at least one inference item constituting the learning inference flow data based on the inference strategy data.
[0245] The learning unit (140) may be configured to include at least one of a query in the second prompt, previous rationales corresponding to the learning rationale data generated in the previous step, rationale strategy data to be applied in the current step, and rationale strategy data to be applied in the next step. Specifically, the second prompt may be configured to include at least one query, previous rationales corresponding to the rationale data generated in the previous step, current step reasoning strategy information applied in the current step, next step reasoning strategy information applied in the next step, and an instruction requesting the generation of learning rationale data to be generated in the current step.
[0246] For example, as illustrated in (a) of FIG. 8c, let us assume that the learning unit (140) generates second learning inference data corresponding to a second inference item among at least one inference item constituting the learning inference flow data. The learning unit (140) may generate a second prompt including at least one instruction requesting the generation of second learning inference data corresponding to the second inference item based on at least one query that is the target of generating the learning data set, first learning inference data, first inference strategy data corresponding to the first inference item, and second inference strategy data corresponding to the second inference item.
[0247] Accordingly, the learning unit (140) may provide the second prompt as an input to the reference large language model (160), and the reference large language model (160) may generate learning inference data corresponding to each of at least one inference item constituting the learning inference flow data based on the inference strategy data included in the second prompt. For example, the reference large language model (160) may generate learning inference data for the current stage according to the inference strategy data corresponding to the current stage, and may perform inference for the next stage by referring to the learning inference data generated in the previous stage so as to maintain the continuity of inference.
[0248] In this way, the learning unit (140) can control the operation of the reference large language model (160) so that learning inference data corresponding to at least one inference item constituting the learning inference flow data is generated step by step using a prompt based on inference strategy data.
[0249] In addition, in the process of generating the above-mentioned training inference data, the training inference data may be generated using inference step role information corresponding to the current inference item. Next, in the present invention, a step of generating a training data set for training a large language model based on the training inference data (or training basis data) may be performed.
[0250] Referring again to FIG. 7d, the learning unit (140) can generate learning inference process data related to generating an answer to the above query using learning inference data. Here, the inference process data is data representing a series of stepwise inference flows performed to generate an answer to the query, and may mean data composed of at least one learning inference data and an answer data to the above query combined (or connected).
[0251] Specifically, the learning unit (140) can generate learning inference process data by sequentially combining at least one generated learning inference data using a reference large language model (160).
[0252] Furthermore, the learning unit (140) can generate learning answer data (730) for at least one query that is the target of generating a learning data set, based on the learning inference data (580) using the reference large language model (160). For example, the learning unit (140) can generate learning inference data corresponding to each of at least one inference item constituting the learning inference flow data (570), and generate a second prompt (720) consisting of at least one instruction requesting the generation of learning answer data (730) based on the learning inference data.
[0253] Furthermore, the reference large language model (160) may generate learning inference process data corresponding to at least one query that is the target of generating the learning data set according to the second prompt (720). At this time, the learning inference process data may be configured to include learning inference data and learning answer data (730) for at least one query.
[0254] As an example, the reference large language model (160) can generate learning inference data (580) corresponding to multiple inference items sequentially, and then generate learning answer data for at least one query based on the sequentially generated learning inference data. Furthermore, the reference large language model (160) can combine the learning inference data and the answer data into a single data sequence to generate learning inference process data corresponding to at least one query that is the target of generating a learning data set.
[0255] Furthermore, the learning unit (140) may generate at least one learning data set for the learning of the large language model (131). At this time, the learning data set may be configured to include at least one query that is the target of generating the learning data set and learning inference process data corresponding to each of the at least one query.
[0256] Meanwhile, the learning unit (140) can select and specify (or filter) the learning data set. Specifically, the learning unit (140) can compare the learning answer data included in the learning inference process data with the correct answer data corresponding to at least one query. Furthermore, based on the comparison result, the learning unit (140) can generate the learning data set to include specific learning inference process data satisfying a preset filtering condition and specific queries corresponding to the specific learning inference process data. Here, the filtering condition may refer to a criterion for determining the accuracy of the generated learning inference process data. In the present invention, the filtering condition can be set in a wide variety of ways. For example, the filtering condition may be set based on whether the learning answer data included in the learning inference process data matches the correct answer data corresponding to at least one query (or whether the learning answer data corresponds to the correct answer data within a preset tolerance range).
[0257] For example, as illustrated in (b) of FIG. 8c, the learning unit (140) may generate a prompt (820, or a fourth prompt) that includes at least one instruction requesting to determine whether the generated learning answer data satisfies a preset filtering condition. Specifically, the learning unit (140) may configure the fourth prompt using a prompt generation unit to include at least one of at least one query that is the target of generating the learning data set, learning inference process data generated by the reference large language model (160), learning answer data, and correct answer data corresponding to the at least one query. As an example, the fourth prompt may include at least one instruction requesting to determine whether the learning answer data and the correct answer data corresponding to the at least one query satisfy a preset filtering condition.
[0258] Furthermore, the learning unit (140) may provide (or process) the fourth prompt as input to the reference large language model (160), and the reference large language model (160) may determine whether the learning answer data satisfies a preset filtering condition based on the fourth prompt. For example, the reference large language model (160) may determine whether the learning answer data logically matches the correct answer data and may generate a learning data set to include specific learning inference process data that satisfies the preset filtering condition and specific queries corresponding to the specific learning inference process data.
[0259] In this way, the learning unit (140) can improve the quality of the learning data set by excluding inaccurate learning inference process data from the learning data set by performing a verification process through comparison with the correct answer data for the generated learning inference data or learning answer data. In addition, the learning data verification and filtering process according to the present invention is not limited to a specific prompt structure or judgment method, and can be performed by various methods capable of verifying the accuracy (or reliability) of the generated learning data.
[0260] In one embodiment, filtering conditions may be set by applying a combination of multiple quality criteria. For example, filtering conditions may include (1) whether the training answer data and the correct answer data match (accuracy criterion), (2) whether the number of inference steps included in the training inference process data is within the allowable range of the reference inference scale data (inference scale criterion), (3) whether the training inference data corresponding to each inference item is logically consistent (consistency criterion), (4) whether the training inference data contains factual errors or hallucinations (factuality criterion), etc.
[0261] Specifically, the learning unit (140) may include a directive in the fourth prompt requesting an evaluation for each of the plurality of quality criteria, and the standard large language model (160) may calculate a score of 0 to 10 for each criterion. Furthermore, the learning unit (140) may include only the learning inference process data in the final learning data set in which the weighted sum score for the plurality of quality criteria is greater than or equal to a preset threshold (e.g., 7.0 points or higher). Through this multi-criteria filtering approach, the quality of the learning data can be managed more systematically.
[0262] In the above, the process of generating a training data set for training a large language model (131) to be trained has been described. Below, the process of training a large language model to be trained using the generated training data set will be described.
[0263] Next, in the present invention, a step of training the large language model to generate an answer to a user query received from a user terminal using a training data set may be performed.
[0264] The learning unit (140) can train a large language model (131) to generate an answer to a user query (510) received from a user terminal using the generated learning data set. As previously described, the learning data set may include at least one query and learning inference process data corresponding to said query, and the learning inference process data may include learning inference flow data related to the generation of an answer corresponding to the query, learning inference data corresponding to at least one inference item constituting said learning inference flow data, and learning answer data generated based on the learning inference data.
[0265] Specifically, the learning unit (140) can perform learning by using at least one query and learning inference process data corresponding to the query as learning samples, so that the large language model (131) to be learned learns the inference flow included in the learning inference process data during the process of generating an answer to the query. In the present invention, the learning method of the large language model (131) can be very diverse. For example, the learning unit (140) can provide at least one query as input data and set the learning inference process data corresponding to the at least one query as target output data to train the large language model (131) in a supervised learning manner. Accordingly, the large language model (131) can be trained to generate an answer to the query according to the inference flow included in the learning inference process data.
[0266] Specifically, the learning unit (140) can perform model learning by updating the parameters of the large language model (131) to be learned using a learning data set. Here, the parameters of the large language model (131) include at least one of weight values, bias values, embedding parameters, vision encoder parameters, text encoder parameters, patch embedding parameters, a query / key / value (Q / K / V) matrix of a multi-head attention module, linear transformation coefficients of a feedforward network (FFN), layer normalization coefficients, weights of an attention combination module, an output matrix for calculating token generation logit, scale coefficients and offset coefficients after the activation function, a weight matrix of a multimodal combination module combining image features and text features, a position embedding, a segment embedding, a representation vector of a class token and an image patch token, a learning rate of the vision language model, a dropout rate, and a regularization coefficient. It can mean all learnable parameters.
[0267] For example, the learning unit (140) may provide at least one query included in the learning data set as input to the large language model (131), calculate a loss function based on the difference between at least one of the learning inference flow data, learning inference data, and learning answer data included in the learning inference process data and the output result of the large language model (131), and adjust the parameters of the large language model by performing a backpropagation algorithm based on the loss function. As an example, the learning unit (140) may fine-tun the large language model using the learning data set. In this case, the learning unit (140) may use a pre-trained large language model as an initial model and fine-tune the large language model (131) to generate answers according to a step-by-step inference flow for user queries by additionally training it using the learning data set.
[0268] As another example, the learning unit (140) can perform instruction-based learning (instruction tuning) using a learning data set. For example, the learning unit (140) can train a large language model (131) by configuring at least one query included in the learning data set as input data in the form of an instruction, and setting learning inference process data corresponding to at least one query as response data for the instruction. Accordingly, the large language model (131) can be trained to generate inference process data including inference flow data, inference data, and answer data corresponding to the query, even when various types of queries are received from the user.
[0269] Additionally, the learning unit (140) can perform step-wise reasoning training using a learning data set. For example, the learning unit (140) can train a large language model (131) to generate sequential outputs of reasoning data for each of the multiple reasoning items included in the learning reasoning flow data, and thereby the large language model (131) can be trained to perform reasoning step-wise during the process of generating answers to a query.
[0270] Additionally, the learning unit (140) may apply various forms of learning methods using the above learning data set. For example, the learning data set may consist of a set of learning samples composed of a single query and a single learning inference process data corresponding thereto, and the learning unit (140) may train a large language model (131) using a batch training or mini-batch training method using multiple learning samples. Additionally, the learning data set may include multiple queries and learning inference process data corresponding to various domains, and the learning unit (140) may perform training using multiple learning samples so that the large language model generates answers corresponding to queries of various domains based on the inference process data.
[0271] In this way, the learning unit (140) can train the large language model (131) using a learning data set that includes at least one query and learning inference process data corresponding to the query, thereby training the large language model to generate an answer including inference process data for a user query received from a user terminal. Furthermore, the method of training the large language model according to the present invention is not limited to a specific learning method or learning procedure, and can be performed using various learning algorithms or learning frameworks for training the large language model.
[0272] In the foregoing, a learning method for a large language model according to the present invention has been described in detail. As described above, the learning method according to the present invention trains a model to generate step-by-step inference data based on inference flow data and inference items corresponding to a query, and the trained model is configured to perform step-by-step inference using the same inference flow data and inference items during the inference stage.
[0273] Meanwhile, the learning method of a large language model according to the present invention can be applied to various special domains. In one embodiment, for learning a large language model specialized in a coding domain, the learning inference flow data may be composed of inference items such as ① problem understanding and requirements analysis, ② algorithm design and data structure selection, ③ pseudocode writing, ④ code implementation, ⑤ test case design and verification, ⑥ time / space complexity analysis, and ⑦ edge case processing. A large language model (131) trained with such a coding-specialized inference flow can possess the ability to systematically perform the entire problem-solving process, going beyond simply generating code.
[0274] In another embodiment, for training a query containing multimodal input, the training inference flow data may include inference items corresponding to various input modalities such as images, text, and voice. For example, when X-ray images and patient symptom text are input together in a medical image analysis domain, the inference flow data may be configured in the order of ① image feature analysis (abnormal region detection), ② text symptom analysis, ③ integrated analysis of image and text information, ④ differential diagnosis, and ⑤ derivation of a diagnostic conclusion. Through this, a large language model equipped with multimodal inference capabilities that integrally process multiple input modalities can be trained.
[0275] Hereinafter, a method for generating answers according to the present invention will be described in detail together with FIGS. 9a to 9c and FIGS. 10a to 10c. FIGS. 9a to 9c are conceptual diagrams for explaining a method for generating answers using a large language model according to the present invention, and FIGS. 10a to 10c are conceptual diagrams for explaining an embodiment of providing an answer to a query generated by a large language model according to the present invention.
[0276] The answer generation method according to the present invention is largely composed of (1) a query reception step (S610), (2) an inference flow data generation step (S620), (3) an inference data acquisition step (S630), (4) an answer acquisition step (S640), and (5) an answer provision step (S650). Each step is performed in an organically linked manner centered around a pre-trained large language model (131), and some steps may be repeated or adaptively modified depending on the characteristics of the query.
[0277] In particular, the answer generation method of the present invention is distinguished from conventional direct response methods in that, unlike the conventional direct response method, it does not generate an answer to a query immediately, but rather first explicitly constructs a thought structure (i.e., inference flow data) for answer generation, and then generates an answer by performing stepwise inference based on it. This explicit thought structuring approach is a key mechanism for systematizing the inference process of large language models and improving the quality and reliability of answers to complex queries.
[0278] In the present invention, a step of receiving a query from a user terminal may be performed (S610, see FIG. 6). In addition, in the present invention, a step of generating inference flow data (or thought structure data) having a plurality of inference items for the query may be performed (S620, see FIG. 6).
[0279] In this case, the step of generating inference flow data (S620) may be a step of generating structured inference flow data including multiple inference items in response to the query. The inference flow data may be generated as a structured data structure representing the relationships between multiple inference items, rather than simple text information. For example, the inference flow data may include metadata to express the precedence, dependency, or validation relationships between each inference step, thereby enabling the system to structurally manage the inference process for query resolution.
[0280] As illustrated in FIG. 9a, the control unit (130) can receive a query (510) from a user terminal (10). At this time, the query may include at least one of multimodal input data consisting of text, images, videos, voice, and combinations thereof. Specifically, the control unit (130) can receive the query through at least one service page provided to the user terminal (10). Here, the service page according to the present invention may refer to at least one user interface provided to provide an answer to the query. For example, the service page may include a search window or an input window for receiving a query from a user, and may include an area for displaying an answer corresponding to the input query in the form of text, images, graphic objects, or a combination thereof.
[0281] The types, configurations, layout structures, and display forms of information included in the service page according to the present invention can be set in a wide variety of ways and are not limited to specific information configurations or screen configuration methods. For example, the service page may be configured to include an input area for entering a query, an output area for displaying an answer corresponding to the query, and an additional information area for providing additional information related to the answer, but is not limited thereto. Furthermore, the arrangement method, display order, visual representation form, or interface configuration method of the information displayed on the service page can also be implemented in various ways and may be modified or applied within the scope that does not deviate from the technical concept of the present invention.
[0282] As illustrated in FIG. 9a, the control unit (130) can receive a query (510) from a user terminal (10) through at least one page. Furthermore, the control unit (130) can provide (or process) the received query (510) as input to a pre-trained large language model (131). At this time, the pre-trained large language model (131) may refer to a large language model trained to generate inference process data corresponding to the input query and to generate an answer to the query based on the inference process data.
[0283] In the present invention, the large language model (131) is described as being generated using a training data set created by a large language model training method, but is not limited thereto, and the large language model (131) may include a model that has been trained or generated in various ways. For example, the large language model (131) may include a pre-trained model that has been trained using large-scale text data in advance, and may be a model generated by further fine-tuning the pre-trained model using the training data set according to the present invention.
[0284] Additionally, the large language model (131) may be a model that has been trained from the beginning using a training data set according to the present invention, or it may be a model that has been created by performing additional training or retraining based on a model trained using a different data set.
[0285] Accordingly, the large language model (131) can be interpreted as a concept that includes all types of artificial intelligence models capable of generating inference process data corresponding to a query and generating an answer to the query based on said inference process data, and the present invention is not limited to a specific model structure or learning method.
[0286] Specifically, as illustrated in FIG. 9b, a large language model (131) can generate inference flow data (520) related to generating an answer to a query (510). As previously described, the inference flow data may refer to data configured according to a preset order of at least one inference item performed to generate an answer to a query. In this case, the inference item (521, 522) may refer to individual inference unit information performed to derive an answer to a query.
[0287] In this case, the inference flow data may be generated based on an inference step sequence (or thought step sequence) for the query. For example, the inference step sequence may be a data structure representing the order of multiple inference steps for understanding and solving the query. The inference step sequence may include at least one of a problem interpretation step, an access setting step, a calculation step, and a verification step, and the composition or order of these steps may be changed depending on the type of query or the difficulty of the problem. The inference step sequence may be represented in various forms of data structures, such as a tree structure, a graph structure, or a sequential list structure.
[0288] In addition, the above inference step sequence includes the plurality of inference items, and the process of generating the inference data can generate the inference data corresponding to each of the plurality of inference items for each of the above inference step sequences.
[0289] The above inference items may represent intermediate inference processes or intermediate results performed at each inference stage, and inference data may be generated corresponding to each of the plurality of inference items. For example, for an inference item corresponding to the problem interpretation stage, inference data including the result of semantic analysis of the query may be generated, and for an inference item corresponding to the calculation stage, inference data including an intermediate calculation result or calculation process may be generated. The plurality of inference data generated in this manner may be sequentially combined or structured to form inference process data representing the entire inference process for the query.
[0290] Meanwhile, the control unit (130) may generate inference flow data that includes at least one inference item related to the domain data based on the domain data corresponding to the query received from the user terminal (10). For example, the control unit (130) may identify the domain data corresponding to the query (510) through semantic analysis of the query (510) received from the user terminal (10). There may be a wide variety of methods for identifying the domain data corresponding to the query (510) through semantic analysis of the received query (510) according to the present invention, and the present specification is not limited to any type or method as long as it is a method capable of identifying the domain data corresponding to the query (510) through semantic analysis of the received query (510).
[0291] The control unit (130) according to the present invention may further include at least one of a module and an algorithm that perform the function of specifying domain data corresponding to the query (510) through semantic analysis of the received query (510). For example, the control unit (130) may analyze the semantic features of the query (510) using a pre-trained natural language processing model and classify the domain to which the query (510) belongs based on the analysis results. For example, the natural language processing model may include a transformer-based language model, a BERT (Bidirectional Encoder Representations from Transformers)-based classification model, a GPT-family language model, a RoBERTa model, or other deep learning-based text classification models, but is not limited thereto.
[0292] As another example, the control unit (130) may specify the domain of the query (510) using a machine learning-based classification algorithm. For example, the classification algorithm may include a Support Vector Machine (SVM), Random Forest, Logistic Regression, Naive Bayes, k-Nearest Neighbor, or other text classification-based algorithms. Additionally, the control unit (130) may specify the domain of the query (510) using keyword-based analysis, topic modeling, or semantic similarity-based analysis. For example, the topic modeling method may include Latent Dirichlet Allocation (LDA), Non-negative Matrix Factorization (NMF), or other topic analysis techniques, and the semantic similarity-based analysis may be performed using cosine similarity, inner product similarity, Euclidean distance, or other vector similarity calculation methods.
[0293] In this way, the control unit (130) can identify domain data corresponding to a query (510) received from a user terminal (10) using various artificial intelligence models, natural language processing technologies, or machine learning algorithms, and the present invention is not limited to specific semantic analysis methods or domain classification algorithms.
[0294] The control unit (130) may provide a query (510) received from a user terminal (10) and domain data corresponding to the query together as input to a pre-trained large language model (131). Furthermore, the large language model (131) may generate inference flow data (520) to include at least one inference item related to the domain data based on the domain data corresponding to the query received from the user terminal (10). For example, if the query (510) corresponds to a mathematical domain, the control unit (130) may generate inference flow data (520) to include inference items related to the mathematical domain, such as problem condition analysis, formula setting, calculation execution, and result verification. As another example, if the query (510) corresponds to a legal domain, the control unit (130) may generate inference flow data (520) to include inference items related to the legal domain, such as verification of relevant laws, analysis of factual relationships, review of legal requirements, and derivation of conclusions.
[0295] The method for specifying domain data and the configuration method of inference items corresponding to the domain according to the present invention can be set in a wide variety of ways and are not limited to a specific domain classification method or inference structure.
[0296] In the present invention, a step of obtaining inference data (or basis data) corresponding to at least one of a plurality of inference items may be performed (S630, see FIG. 6). At this time, the inference data may be generated corresponding to each of the plurality of inference items.
[0297] Referring again to FIG. 9b, the control unit (130) can generate inference flow data (520) corresponding to a query (510) received from a user terminal (10) using a pre-trained large language model (131). Furthermore, the control unit (130) can generate inference data (910) corresponding to each of at least one inference item constituting the inference flow data (520) using the pre-trained large language model (131). Specifically, the control unit (130) can control the pre-trained large language model (131) to sequentially generate inference data (911, 912) according to a pre-set order for at least one inference item (521, 522) constituting the inference flow data (520). Here, the pre-set order may refer to the logical processing order of an inference procedure performed to generate an answer to a query.
[0298] The pre-set order according to the present invention is not limited to a specific sequential structure and can be configured in various forms depending on the type of query or domain. For example, the inference items may be configured in a stepwise structure arranged sequentially, or in a structure where some inference items are executed in parallel or selectively according to specific conditions. Therefore, the pre-set order of the inference items according to the present invention can be configured in various forms and is not limited to a specific inference structure or processing order.
[0299] For example, the large language model (131) can first generate first inference data (911) corresponding to the first inference item (521) based on the fact that the order of the first inference item (521) constituting the inference flow data (520) related to the generation of an answer to a query (510) is the first priority. Furthermore, the large language model (131) can generate second inference data (912) corresponding to the second inference item (522) based on the fact that the order of the second inference item (522) constituting the inference flow data (520) related to the generation of an answer is the second priority.
[0300] Specifically, the control unit (130) can control the large language model (131) to generate inference data corresponding to the next set order of inference items using at least one inference data corresponding to the inference item that is prior in a preset order. The large language model (131) can sequentially generate inference data corresponding to the next order of inference items by referring to at least one inference data corresponding to the inference item that is prior in a preset order. For example, the control unit (130) can generate a prompt (or a fifth prompt) including an instruction requesting the large language model (131) to generate a second inference data corresponding to a second inference item by including the first inference data as input information, based on the prompt generation unit, after the large language model (131) has generated a first inference data corresponding to a first inference item.
[0301] Specifically, the prompt may be configured to include an instruction requesting the generation of second inference data (912) corresponding to the second inference item (522) based on a query (510), first inference data (911), and second inference item (522) received from a user terminal (10). Accordingly, the large language model (131) may generate second inference data (912) corresponding to the second inference item (522), which is set to the next order of the first inference item, by referring to the query (510), first inference data (911), and second inference item (522) received from the user terminal (10).
[0302] At this time, for the large language model (131) to generate inference data, the inference data corresponding to the inference item with a preset order that is referenced is not limited to the inference data generated in the immediately preceding order, and may include at least some or all of the inference data corresponding to a plurality of inference items with a preset order that is preceding.
[0303] For example, in the process of generating inference data corresponding to the next inference item, the large language model (131) may refer only to the inference data generated immediately prior, may refer together to multiple inference data generated earlier in time, or may be configured to refer only to some inference data selected according to a preset standard. As such, in the present invention, the range, number, or selection method of the inference data referenced by the large language model (131) can be set in a wide variety of ways and is not limited to a specific reference method.
[0304] Meanwhile, the inference data (910) generated by the large language model (131) may include at least one inference transition information (920). In this case, the inference transition information (920) may refer to information indicating that an inference transition event occurred during the process of generating inference data to generate an answer to a query. Here, the inference transition event may refer to an event that includes at least one of reviewing and changing the inference data that was previously performed during the process of generating inference data to generate an answer to a query.
[0305] Specifically, the inference transition information (920) may include information indicating a point where the content of the generated initial inference data is changed by verifying the validity of at least one inference data generated earlier in time or by recognizing the possibility of an error that occurred during the existing inference process. For example, the inference transition information (920) may include text such as “wait”, “let’s reconsider”, “okay”, “verify”, “check again”, “however”, “but”, “rethink”, etc.
[0306] In one embodiment, the reasoning transition information may be composed of a meta-reasoning expression in the form of natural language. For example, if a large language model (131) determines that previously generated reasoning data is incorrect at a specific reasoning step, it may generate a reasoning transition expression within the reasoning data, such as "I will review this approach again" or "I am reconfirming because there may be an error in the previously calculated result." A large language model (131) trained with training data containing such reasoning transition expressions can exhibit self-correction reasoning ability to actively detect and correct its own errors during the reasoning process.
[0307] Furthermore, the inference switching information may also be configured as an inference restart signal in a binary form. Specifically, the control unit (130) can induce the large language model (131) to restart inference under specific conditions (e.g., detection of a contradiction in an intermediate calculation result, detection of a mismatch in a precondition, etc.) by specifying the conditions for generating the inference switching signal in the prompt. Through this, it is possible to prevent the inference error from propagating and the quality of the final answer from degrading.
[0308] The inference conversion information according to the present invention is not limited to the expressions exemplified above, and may include various forms of expressions, symbols, sentence structures, or semantic signals capable of indicating a process in which inference is converted to review, modify, or supplement existing inference data during the inference data generation process. Accordingly, the expression method, configuration form, or type of information included in the inference conversion information according to the present invention can be set in a wide variety of ways and is not limited to a specific expression method or structure.
[0309] The control unit (130) according to the present invention can control the operation of a large language model (131) so that inference switching information is included in the inference data according to preset conditions. Specifically, the control unit (130) can use a prompt generation unit to generate a prompt (or a sixth prompt) that includes an instruction requesting that inference data including inference switching information be generated during the process of generating inference data.
[0310] Furthermore, the control unit (130) may provide a prompt as an input to the large language model (131), and accordingly, the large language model (131) may operate to generate inference data including inference transition information when a preset condition is satisfied. For example, the preset condition may be set as a condition satisfied based on at least one of the cases where a preset number of steps or more is performed during the inference data generation process, where a preset number of tokens or more is generated, and where a specific inference item is performed, but is not limited thereto. In addition, the preset condition may be set to include various situations where inference transition is determined to be necessary during the inference data generation process, and the generation condition, application method, and control method of the inference transition information according to the present invention may be implemented in various forms and are not limited to a specific condition setting method.
[0311] Next, in the present invention, the inference data (or, basis data) and the query may be provided as inputs to a pre-trained large language model, and a step of obtaining an answer to the query from the large language model may be performed (S640, see FIG. 6).
[0312] The control unit (130) can generate inference process data related to generating the answer to the query using inference data. As illustrated in FIG. 9c, the control unit (130) can generate inference process data (530) related to generating the answer to the query based on at least one generated inference data (910) using a large language model (131). Here, the inference process data is data representing a series of stepwise inference flows performed to generate an answer to the query, and may mean data composed of at least one inference data (910) and an answer data to the query (510) sequentially combined (or connected).
[0313] Specifically, the control unit (130) may provide inference data corresponding to at least one inference item constituting the inference flow data and a query (510) received from the user terminal (10) as input to a pre-trained large language model. The large language model (131) may sequentially combine at least one generated inference data (910) to generate inference process data (530). Here, sequentially combining the inference data (910) may mean that at least one inference data is connected according to a preset order to form a single continuous inference process data in order to generate an answer to the query.
[0314] The sequential combination of inference data according to the present invention is not limited to a form in which text is simply concatenated, but may include cases where each inference data is composed of a structural unit corresponding to an inference item and arranged to form a stepwise structure. Accordingly, the method of sequential combination, arrangement structure, or configuration form of inference data according to the present invention can be set in a wide variety of ways and is not limited to a specific combination method or expression format.
[0315] Specifically, the large language model (131) can configure the inference process data such that the inference data generated in the next step is connected after the inference data generated earlier in time. Furthermore, when the generation of at least one inference data corresponding to at least one inference item constituting the inference flow data is completed, the large language model (131) can generate answer data for the query (510) based on the generated at least one inference data. Here, the answer data may refer to data generated by the large language model (131) to provide an answer to the query (510) provided to the user terminal (10).
[0316] Specifically, the answer data may include an answer (930) to a query (510) derived based on the generated inference process data, and may be configured in the form of text, images, charts, tables, voice, or a combination thereof. Furthermore, the control unit (130) may obtain at least a portion of the inference process data, including at least one inference data and an answer data to a query generated based on at least one inference data, from the large language model (131) as an answer (930) to the query. At this time, the answer data may be identical to the answer (930) to the query provided to the user terminal (10), but is not limited thereto, and may include at least one of intermediate result or candidate answer data generated during the internal processing of the large language model (131) to generate an answer (930) to the query (510).
[0317] Accordingly, the answer data according to the present invention may include not only data that directly represents an answer to a query (510), but also various forms of data related to an answer to a query that can be utilized in the process of generating an answer to a query (510) or used in subsequent processing, and the method of expression, configuration, or utilization thereof is not limited to a specific format. For example, when a first inference data corresponding to a first inference item is generated and a second inference data corresponding to a second inference item is generated, the large language model (131) may sequentially combine (or connect) the first inference data and the second inference data according to a preset order to form a single inference process data. Furthermore, the large language model (131) may generate answer data for a query received from a user terminal (10) based on at least one inference data including the first inference data and the second inference data.
[0318] Furthermore, the large language model (131) can generate inference process data (530) including at least one inference data and answer data for a query as an answer (930) to a query. Alternatively, the large language model (131) can generate the answer data for a query among the inference process data as an answer (930) to a query.
[0319] Additionally, the large language model (131) may generate an answer (930) to a query by summarizing or reorganizing at least one of the inference data and answer data to a query included in the inference process data (530). Additionally, the large language model (131) may generate an answer (930) to a query (510) by combining or rearranging at least one of the inference data and answer data included in the inference process data (530).
[0320] As another example, a large language model (131) may verify the logical validity of the inference data and answer data included in the inference process data (530) and then generate an answer (930) to a query (510) based on the verified result. Accordingly, the answer (930) to a query according to the present invention may include not only a form in which the inference data and answer data included in the inference process data (530) are provided as is, but also various forms of responses generated through reconstruction, combination, summarization, or verification processes based on the inference process data, and the processing method for generating an answer to a query (510) is not limited to a specific form.
[0321] Next, in the present invention, a step of providing an answer to a user terminal may be performed (S650, see FIG. 6). To this end, the answer data may be transmitted to the user terminal.
[0322] The control unit (130) can provide an answer (930) to a query to a user terminal. Specifically, the control unit (130) can provide an answer (930) to a query to a user terminal through at least one service page (1000). At this time, the service page (1000) is a user interface displayed on the output unit (or display unit) of the user terminal (10), and may refer to one or more screen configurations for providing information related to an answer to a query and the generation of an answer. For example, the control unit (130) can provide an answer (930) to a query received through the service page (1000) to the user terminal (10) by displaying it in a form including text, a table, a chart, an image, video, audio, or a combination thereof.
[0323] Specifically, the control unit (130) may provide at least a portion of the inference process data (530) as an answer (930) to the query through the service page (1000). As illustrated in FIG. 10a, the control unit (130) may provide at least one inference item and inference data related to the generation of an answer as an answer (930) to the query through at least one service page (1000). For example, the control unit (130) may provide (or output) to the service page (1000) at least one inference item (521a) constituting the inference flow data related to the generation of an answer to a query (510a) received from a user terminal (10), and inference data (910a) corresponding to said at least one inference item (521a).
[0324] At this time, the control unit (130) may receive (or detect) a modification event for at least one inference item that constitutes inference flow data related to the generation of an answer to a query from the user terminal (10). For example, the control unit (130) may provide a modification icon (940) corresponding to each of the at least one inference item to the service page (1000). Furthermore, when the control unit (130) receives (or detects) user input regarding the modification icon (940) from the user terminal (10), it may provide a modification interface to the user terminal (10) that allows the modification of at least one inference item. Furthermore, the control unit (130) may receive a modification inference item related to at least one inference item as the modification event through the modification interface.
[0325] As a specific example, assume a case where a user inputs a query to "analyze the current status of environmental regulations in Korea." When the initial inference flow data generated by the large language model (131) consists of ① background and purpose, ② major environmental laws, ③ current status of regulations, and ④ future tasks, the user can input a modification event through the service page (1000) to "subdivide inference item ② into laws and enforcement decrees related to air, water, and soil pollution." The control unit (130) receives this modification event, generates modified inference flow data, and inputs it into the large language model (131) to generate a more detailed environmental regulation analysis answer according to the modified inference structure.
[0326] In this case, the control unit (130) can automatically generate sub-inference items (air-related laws, water-related laws, soil-related laws) dependent on the modified inference item (②) and integrate them with the remaining existing inference items (①, ③, ④) to form modified inference flow data. At this time, the computational cost required for regenerating the entire answer can be reduced by reusing parts of the inference results prior to modification that are not affected by the modified inference item (e.g., inference data regarding the background and purpose of ①).
[0327] Furthermore, the control unit (130) can generate modification inference flow data related to the generation of an answer to a query based on a modification event received from the user terminal (10). For example, if the control unit (130) receives a modification event for a specific inference item from the user terminal (10) through the service page (1000), it can generate modification inference flow data to include at least one modification inference item modified according to the modification event.
[0328] The control unit (130) can provide (or process) the generated modification inference flow data as input to a pre-trained large language model (131). Furthermore, the large language model (131) can generate modification inference data corresponding to at least one modification inference item. The control unit (130) receives a modification answer generated based on the modification inference data from the large language model and can provide a modification answer for the query to the user terminal (10).
[0329] Meanwhile, as previously described, the control unit (130) can generate inference flow data that includes at least one inference item related to the domain data based on the domain data corresponding to the query received from the user terminal (10). For example, the control unit (130) can identify the domain data corresponding to the query (510) through semantic analysis of the query (510) received from the user terminal (10). Furthermore, the control unit (130) can control the operation of the large language model (131) to generate inference process data for the query based on the inference flow data corresponding to the identified domain.
[0330] Since the method for determining domain data corresponding to a query (510) through semantic analysis of a query (510) received according to the present invention has been explained above, it will be omitted to avoid duplication of explanation.
[0331] For example, as illustrated in FIG. 10b, if a query (510b) received from a user terminal (10) is related to a first domain data (e.g., “legal field”), the control unit (130) can generate a first inference flow data corresponding to the first domain data. Furthermore, the control unit (130) can generate a first inference process data (530a) including at least one inference item constituting the first inference flow data by using a large language model (131). The control unit (130) can provide at least a portion of the first inference process data as an answer to the query through a service page (1000).
[0332] As a specific example, the process of generating an answer to a query belonging to a legal domain (e.g., 'What are the requirements for claiming damages in case of breach of contract?') is described. In this case, the control unit (130) can specify the domain of the query as 'law' and generate legal domain-specific inference flow data corresponding thereto. The generated inference flow data may be composed in the order of, for example, ① verification of relevant legal provisions (Articles 390, 393, etc. of the Civil Code), ② examination of whether a contract is formed, ③ verification of the fact of default, ④ examination of the occurrence of damages and causal relationship, ⑤ calculation of the scope of damage compensation, and ⑥ derivation of a conclusion. The large language model (131) can generate inference data step by step according to this legal domain-specific inference flow and finally generate a legally accurate and systematic answer.
[0333] As another embodiment, the process of generating an answer to a query belonging to a medical domain (e.g., 'What is the blood sugar management method for patients with type 2 diabetes?') is described. In this case, the control unit (130) may specify the domain of the query as 'medical' and generate medical domain-specific inference flow data corresponding thereto. The generated inference flow data may be composed in the following order, for example: ① confirmation of disease overview and pathophysiology, ② review of current treatment guidelines, ③ review of dietary and exercise therapies, ④ review of drug treatment options, ⑤ review of blood sugar monitoring methods, ⑥ confirmation of precautions and contraindications, and ⑦ derivation of comprehensive recommendations. In this way, the present invention can provide high-quality answers that reflect domain-specific expertise by dynamically generating an optimized inference structure according to the domain of the query.
[0334] Furthermore, the process of generating answers for complex queries belonging to the mathematical domain (e.g., 'When 3x + 2y = 12 and x - y = 1, find the values of x and y and explain the geometric meaning of the solution') is described. In this case, the inference flow data may be composed of the following steps: ① analysis of the problem type (system of equations), ② setting of the solution strategy (substitution or elimination method), ③ arranging the expression for x or y, ④ calculating variables by substitution, ⑤ verification of the calculation result (substitution into the original equation), ⑥ geometric interpretation (intersection point of two lines), and ⑦ writing the final answer. The large language model (131) sequentially generates inference data corresponding to each inference item, accumulates intermediate calculation results, and derives the final answer based on them. At this time, if a calculation error is found in the verification step (inference item ⑤), the error can be automatically corrected by returning to the previous step and performing recalculation through the inference transition information.
[0335] For example, as illustrated in FIG. 10c, if a query (510c) received from a user terminal (10) is related to second domain data (e.g., “medical field”), the control unit (130) can generate second inference flow data corresponding to the second domain data. Furthermore, the control unit (130) can generate second inference process data (530b) including at least one inference item constituting the second inference flow data by using a large language model (131). The control unit (130) can provide at least a portion of the second inference process data as an answer to the query through a service page (1000).
[0336] In this way, the control unit (130) can control the operation of the large language model (131) so that inference process data suitable for the characteristics of the query is generated by generating different inference flow data based on domain data corresponding to the query. Accordingly, the control unit (130) can improve the accuracy and reliability of the answer to the query by generating inference process data according to an inference procedure specialized for the domain to which the query belongs.
[0337] In the foregoing, the answer generation method according to the present invention has been described in detail. Below, with reference to FIG. 11, the performance of the answer generation system (100) according to the present invention will be briefly described. FIG. 11 is a conceptual diagram for explaining the performance of a pre-trained large language model according to the present invention.
[0338] The answer generation system (100) according to the present invention can improve long-term reasoning ability and reasoning accuracy for answer generation by generating inference flow data related to the generation of answers to a query, generating stepwise inference data according to the inference flow data, and generating an answer to a query based on the generated inference data. For example, a large language model (131) according to the present invention can be trained to generate an inference process performed to generate an answer to a query in a structured form, and a model applying such a training structure can exhibit improved performance in various knowledge and inference evaluation benchmarks compared to a general command-based model.
[0339] Specifically, as shown in Fig. 11, the instruction-based model Llama-3.1-8B-Instruct shows a performance of approximately 22.7 on the GPQA Diamond benchmark, whereas the pre-trained large language model (131, “Llama-3.1-8B-LC”) shows a performance of approximately 36.4, confirming that performance can be significantly improved even at the same model size. In addition, on the MMLU Pro benchmark, the Llama-3.1-8B-Instruct model shows a performance of approximately 43.7, while the pre-trained large language model (131, “Llama-3.1-8B-LC”) shows a performance of approximately 44.5, confirming performance improvement. As another example, the Qwen-2.5-7B-Instruct model shows a performance of about 37.6 on the GPQA Diamond benchmark, while the pre-trained large language model (131, “Qwen-2.5-7B-LC”) shows a performance of about 39.9, and it can be seen that the performance also improves from about 49.9 to about 51.4 on the MMLU Pro benchmark.
[0340] Furthermore, the effect of the learning method according to the present invention may be more pronounced in benchmarks requiring complex multi-stage reasoning. For example, performance improvement of large language models to which the learning method according to the present invention is applied can be confirmed in the MATH benchmark, which evaluates mathematical reasoning ability, and the ARC-Challenge benchmark, which evaluates scientific reasoning ability. This suggests that the stepwise reasoning learning method of the present invention can be universally applied to various problem types requiring multi-stage reasoning, going beyond simple knowledge-based question answering.
[0341] Furthermore, the effectiveness of the answer generation method according to the present invention can also be confirmed in terms of response time (latency). A method of pre-configuring inference flow data and sequentially generating inference data for each inference item can reduce unnecessary repetitive inference or inefficient search processes by structuring the inference process. In one embodiment, when the structured inference method of the present invention is applied, there may be an effect of reducing the average response generation time compared to an unstructured inference method.
[0342] Furthermore, as a result of analyzing the effects of the interactive answer modification function according to the present invention, it can be confirmed that when a user modifies an inference item one or more times, the quality of the final answer is improved compared to the initial answer generated without modification. In particular, in domains requiring specialized knowledge (e.g., law, medicine, finance, etc.), the accuracy and expertise of the answer can be further improved by reflecting the user's domain expertise in the inference flow.
[0343] As seen above, according to the present invention, an answer to a query can be generated according to a stepwise thinking structure. Through this, multiple thinking steps performed during the answer generation process can be structurally unfolded, and by maintaining a consistent thinking process according to the stepwise thinking structure, the reliability and inference performance of generating answers to complex queries can be improved.
[0344] Furthermore, according to the present invention, a large language model can generate an answer to a query requiring multiple thinking stages through a step-by-step thinking process. Through this, even for complex queries, an answer can be generated by reasoning step-by-step, thereby expanding the problem-solving scope and application possibilities of the large language model.
[0345] Furthermore, according to the present invention, an appropriate thinking structure can be controlled to be applied according to the characteristics of the query. Through this, by selectively performing a suitable thinking process according to the domain of the query or the required scale of inference, it is possible to prevent situations where unnecessary excessive inference is performed or insufficient inference is not achieved, and to improve the efficiency and accuracy of generating customized answers to the query.
[0346] Furthermore, according to the present invention, a large language model can be trained using training data configured to generate answers according to a stepwise thinking structure. Through this, the large language model can effectively learn stepwise thinking process patterns during the answer generation process, thereby improving long-term reasoning ability for complex queries and consistency in response generation.
[0347] Furthermore, according to the present invention, training data can be generated configured such that the generation of answers to queries is performed according to a step-by-step thinking structure. Through this, training data including a step-by-step thinking process can be efficiently constructed, and by ensuring diversity and scalability of training data for various problem types and domains, the generalization performance of large language models and their usability in actual service environments can be improved.
[0348] Furthermore, according to the present invention, by applying a processing method based on an inference step sequence to break down the query resolution process into multiple stages, higher inference accuracy can be provided compared to a single-stage inference method. In addition, this stage-based processing structure can reduce the possibility of errors occurring when a large language model processes complex queries.
[0349] Furthermore, according to the present invention, the accuracy of the final answer can be improved by automatically detecting and correcting errors occurring during the initial inference process through a self-correcting inference function using inference conversion information during the inference data generation process.
[0350] Furthermore, according to the present invention, users can directly modify inference items through a service page and receive a modified answer reflecting this in real time, thereby improving the user experience and the level of customization of the answer.
[0351] Meanwhile, the present invention described above can be implemented based on a quantum computer. The present invention implemented based on a quantum computer may include a qubit-based quantum processor and quantum memory, and may include software and hardware interfaces optimized for quantum computation.
[0352] Quantum processors in quantum computers utilize qubits to efficiently process complex operations through parallel computation, quantum entanglement, and quantum superposition, which cannot be performed by the binary bits of classical computers. Quantum processors process data using quantum gates and can provide exponential speed improvements for specific problems.
[0353] Meanwhile, the present invention described above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a computer-readable medium (or recording medium).
[0354] Furthermore, the present invention described above can be implemented as computer-readable code or instructions on a medium on which a program is recorded. That is, the present invention can be provided in the form of a program.
[0355] Meanwhile, computer-readable media include all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
[0356] Furthermore, the computer-readable medium may be a server or cloud storage that includes a storage and is accessible to an electronic device via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage via wired or wireless communication.
[0357] A computer program may reach the system (100) through various suitable transmission mechanisms. The transmission mechanism may be, for example, a computer-readable storage medium, a computer program product, a memory device, a recording medium such as a CD-ROM or DVD, or a product that tangibly embodies the computer program. The transmission mechanism may be a signal configured to reliably transmit the computer program through air or an electrical connection. The answer generation system (100) may propagate or transmit the computer program as a computer data signal.
[0358] Furthermore, references to 'computer-readable storage media,' 'computer program products,' 'computer programs embodied in a tangible form,' etc., or to 'controller,' 'computer,' 'processor,' etc., should be understood to include not only computers with various architectures such as single or multi-processor architectures, sequential (Von Neumann) architectures, or parallel architectures, but also specialized circuits such as Field-Programmable Gate Arrays (FPGAs), Application Specific Circuits (ASICs), signal processing units, and other devices. References to computer programs, instructions, code, etc., should be understood to include software for programmable processors or firmware, such as programmable content for hardware devices, whether it is instructions for a processor or configuration settings for a fixed-function device, gate array, or programmable logic device.
[0359] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, namely a CPU (Central Processing Unit), and no special limitations are placed on its type.
[0360] Meanwhile, the above detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.
Claims
1. Regarding methods performed by a computer, Step of receiving a query; A step of generating thought structure data having at least one inference item for the above query; A step of obtaining basis data corresponding to at least one inference item above; A step of providing the above-mentioned basis data and the above-mentioned query as input to a pre-trained large language model to obtain an answer to the above-mentioned query from the large language model; and A method for generating an answer characterized by including the step of providing the above answer to the user terminal.
2. In Paragraph 1, The above accident structure data is generated based on the inference step sequence for the above query, and A method for generating an answer characterized in that the above inference step sequence includes at least one of a problem interpretation step, an approach setting step, a calculation step, and a verification step.
3. In Paragraph 2, The step of obtaining the above-mentioned basis data is, A method for generating an answer characterized by generating the basis data corresponding to each of the at least one inference item for each of the above inference step sequences.
4. In Paragraph 1, In the step of obtaining the above-mentioned basis data, A method for generating an answer characterized by sequentially generating the basis data according to a pre-set order of at least one inference item constituting the above accident structure data.
5. In Paragraph 4, It further includes the step of generating thought process data for the above query using the above-mentioned basis data, and The step of generating the above accident process data is, A method for generating an answer characterized by sequentially combining the generated basis data to generate the thought process data.
6. In Paragraph 5, The above accident process data includes the above basis data and the answer data to the above query generated based on the above basis data, and In the step of obtaining an answer to the above query from the above large language model, A method for generating an answer characterized by obtaining at least a portion of the above-mentioned thought process data as an answer to the above-mentioned query.
7. In Paragraph 1, The step of generating the above accident structure data is, A method for generating an answer characterized by generating the thought structure data to include at least one inference item related to the domain data, based on the domain data corresponding to the above query.
8. In Paragraph 1, In the step of providing the above answer to the user terminal, A method for generating an answer characterized by providing, through at least one service page, the at least one inference item and the basis data corresponding to the at least one inference item as an answer to the query.
9. In Paragraph 8, When a modification event for a specific inference item is received from the user terminal through the service page, a step of generating modification thought structure data to include at least one modification inference item modified according to the modification event; A step of providing the generated modified thought structure data as input to the pre-trained large language model; A step of generating modification basis data corresponding to at least one modification inference item in the above large language model; A step of receiving a modified answer generated based on the above modification basis data from the above large language model; and A method for generating an answer characterized by further including the step of providing the modified answer to the above query to the user terminal.
10. Memory for storing instructions; and It includes at least one processor electrically connected to the memory, and When the above instructions are executed by the at least one processor, the at least one processor, Receive a query from a user terminal, and Generate thought structure data having at least one inference item for the above query, and Obtaining basis data corresponding to each of the above at least one inference item, and The above-mentioned basis data and the above-mentioned query are provided as input to a pre-trained large language model, and an answer to the above-mentioned query is obtained from the large language model, and An answer generation system characterized by providing the above answer to the above user terminal.
11. In a method performed by a computer, A step of specifying a reference training data set corresponding to at least one query; A step of generating learning thought structure data for the above query using the above reference learning data set; A step of acquiring learning basis data for each of at least one inference item constituting the above-mentioned learning thought structure data; Based on the above-mentioned learning basis data, a step of generating a training data set for training a large language model; and A method for training a large language model, characterized by including the step of training the large language model to generate an answer to a user query using the above training data set.
12. In Paragraph 11, In the step of acquiring the above-mentioned learning basis data, A learning method for a large language model characterized by sequentially generating the learning basis data according to a pre-set order of at least one inference item constituting the learning thought structure data.
13. In Paragraph 12, The method further includes the step of generating learning thought process data for the above query using the above learning basis data, and In the step of generating the above-mentioned learning thought process data, A learning method for a large language model characterized by sequentially combining at least one generated learning basis data to generate the learning thought process data.
14. In Paragraph 13, The above learning thought process data is configured to include the above learning basis data and the above learning answer data for at least one query, and The above training data set is, A method for learning a large language model characterized by being configured to include at least one query and learning thought process data corresponding to each of the at least one query.
15. In Paragraph 11, The step of specifying the above-mentioned reference training data set is, A step of identifying candidate query data corresponding to at least one query in a plurality of basic training data according to a preset similarity criterion; A step of calculating domain similarity between domain data corresponding to at least one query and candidate domain data corresponding to candidate query data; A step of identifying candidate inference scale data corresponding to the above candidate query data and calculating the inference scale similarity between the above candidate inference scale data; and A method for training a large language model, characterized by including the step of specifying a reference query data among the candidate query data based on at least one of the domain similarity and the inference scale similarity, and specifying a reference learning data set including at least one of reference thought structure data and reference inference scale data corresponding to the reference query data.
16. In Paragraph 15, The step of generating the above-mentioned learning thought structure data is, A step of providing the above-mentioned reference query data, the above-mentioned reference thought structure data, the above-mentioned reference inference scale data, and the above-mentioned at least one query as input to a pre-trained reference large language model; and A method for learning a large language model, characterized by including the step of generating the learning thought structure data for at least one query based on the reference learning data set in the reference large language model.
17. In Paragraph 16, In the step of generating the above-mentioned learning thought structure data, Using at least one of the above reference accident structure data and the above reference inference scale data, the number of the at least one inference item is predicted, and A learning method for a large language model characterized by generating the learning thought structure data based on the number of predicted inference items.
18. In Paragraph 11, Using the above at least one query, the pre-generated learning basis data, the first inference item and the second inference item, a prompt is generated to request the generation of the learning basis data, and The above prompt is provided as input to a pre-trained standard large language model along with the above learning thought structure data, and A method for learning a large language model, characterized by generating learning basis data by referring to the second inference item in order to generate first learning basis data corresponding to the first inference item based on the prompt in the large language model above.
19. In Paragraph 13, The step of generating the above training data set is, A step of comparing learning answer data included in the above learning thought process data with correct answer data corresponding to at least one of the above questions; and A method for training a large language model, characterized by including the step of generating a training data set to include specific learning thought process data satisfying preset filtering conditions and specific queries corresponding to said specific learning thought process data, based on the above comparison results.
20. Memory for storing instructions; and It includes at least one processor electrically connected to the memory, and When the above instructions are executed by the at least one processor, the at least one processor, Specify a reference training data set corresponding to at least one query, and Using the above standard training data set, generate learning thought structure data for the above query, and Acquiring learning basis data corresponding to at least one inference item constituting the above-mentioned learning thought structure data, and Based on the above-mentioned learning basis data, a training data set for training a large language model is generated, and A learning system for a large language model characterized by training the large language model to generate answers to user queries using the above-mentioned training data set.
