Method and system for generating answers
The method verifies and evaluates the multi-stage thought structure and final answer using a pre-trained language model, enhancing answer quality and reliability by considering the entire thought process, addressing limitations in existing evaluation methods.
Patent Information
- Application Number
- PCT/KR2026/005923
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2026-05-06
- Filing Date
- 2026-05-08
- Publication Date
- 2026-11-12
AI Technical Summary
Existing evaluation methods for large language models fail to adequately assess the quality of answers by solely focusing on the final answer, neglecting the validity of intermediate inferences and external information utilization, particularly in environments with dynamic or unclear criteria.
A method and system that verifies the multi-stage thought structure and final answer during the query response generation process, utilizing a pre-trained language model to evaluate and generate answers based on thought process data and candidate evaluation results.
Ensures consistent and reliable answer generation by considering the entire thought process, improving answer quality and reliability, especially in complex queries and dynamic environments.
Smart Images

Figure KR2026005923_12112026_PF_FP_ABST
Abstract
Description
Answer generation method and system
[0001] The present invention relates to an answer generation method and system capable of generating an answer to a query. Furthermore, the present invention relates to a language model learning method and system for generating an answer to a query.
[0002] The dictionary definition of artificial intelligence is a technology that realizes human learning, reasoning, perception, and natural language understanding abilities through computer programs. With the advancement of deep learning, the ability of such artificial intelligence to automatically recognize and learn complex patterns has improved, and it is being utilized in various industrial fields.
[0003] In particular, driven by the advancement of artificial intelligence, various language models have been developed. These models have reached a level where they not only recognize text and understand its meaning but also extract and classify information from vast amounts of text-based data, such as documents, and even generate text directly.
[0004] These language models are being actively utilized in various fields. For example, they are used in diverse areas where text-based tasks can be performed, such as search engines, document creation (e.g., resume writing, report writing, posting, etc.), free conversation on various topics, data parsing from given text (e.g., data summarization, classification, etc.), provision of expertise, programming, and converting given sentences into sentences of an appropriate style.
[0005] In this regard, Large Language Models (LLMs) have emerged, which understand and generate human language through prior training on vast amounts of text data. Unlike chatbots, which are manually built and provide only limited answers, LLMs are being utilized as a technology capable of interacting with users in a conversational manner and providing information across a wide range of domains.
[0006] Furthermore, research on Retrieval-Augmented Generation (RAG) technology based on these large language models has recently been underway. This RAG technology is characterized by its ability to complement the knowledge limitations of large language models and reflect the latest information by retrieving information from external documents or databases and generating answers based on it. In particular, agent-based RAG technology is emerging that goes beyond single-stage search and generation structures to include multi-stage search, tool invocation, and inference processes; this technology is evolving into a method of solving problems by forming multi-stage thinking regarding complex queries.
[0007] Meanwhile, as a method to judge the quality of answers generated through such large language model-based RAG technology, an approach that evaluates text similarity based on the final answer is widely used. For example, the accuracy of an answer can be determined by evaluation methods based on indicators such as EM, F1, and ROUGE by comparing the final answer with the correct answer reference. Since this method judges the quality of the answer based solely on the final answer, it has limitations in that it fails to sufficiently reflect the validity of intermediate inferences formed during the answer generation process or whether the answer is based on external information.
[0008] Accordingly, research is being conducted to consider the relationship between intermediate thought processes and information acquired from external sources, and there is growing interest in technical approaches to analyze the correlation between the thought processes formed during the answer generation process and the answers themselves.
[0009] Furthermore, in various industrial environments, data such as patent documents, legal documents, financial data, ERP records, and technical documents may be utilized where the correct answer is not fixed or it is difficult to establish clear criteria; in such environments, evaluation methods different from those based on correct answer data are required.
[0010] Accordingly, there is a need for technology that enables large language models to verify and evaluate both the thought processes generated during the query response generation process and the final answer, and to generate answers based on the evaluation results.
[0011] The present invention aims to provide an answer generation method and system capable of generating an answer by simultaneously verifying the thought structure and the answer during the process of generating an answer to a query.
[0012] Specifically, the present invention aims to provide an answer generation method and system that can verify the multi-stage thought structure and the final answer formed during the process of generating an answer to a query in an Agentic Search Augmentation Generation (Agentic RAG) environment, and generate an answer by reflecting the verification results.
[0013] Furthermore, the present invention aims to provide a method and system for generating answers that can derive answers based on thought structure data and evaluation results by considering the validity of the thought structure and results generated throughout the entire operation process of an agent-based AI (Agentic AI) that performs multiple stages of inference trajectory and tool utilization.
[0014] Furthermore, the present invention aims to provide a language model training method and system capable of training a large language model to generate an answer by simultaneously verifying the thought structure and the answer during the process of generating an answer to a query.
[0015] Specifically, the present invention aims to provide a language model learning method and system capable of generating a learning data set in which the thought structure and the answer in the process of generating an answer to a query are simultaneously verified.
[0016] Furthermore, the present invention aims to provide a language model training method and system for training a large language model to generate answers by reflecting the evaluation results of the thought structure data in the learning process, by reflecting the validity of the evaluation results of the multi-stage thought structure and answers performed in an agentic AI environment.
[0017] To solve the problem described above, the present invention proposes a method and system for providing an answer to a query by utilizing a pre-trained large language model that evaluates a multi-stage thought structure formed during the process of generating an answer to a query together with the final answer, and generates the final answer based on the evaluation result. The answer generation method according to the present invention, which is performed by a computer, may include the steps of receiving at least one query; inputting the query into at least one pre-trained language model; obtaining thought process data and answer candidate data regarding the query from the language model; inputting the thought process data and the answer candidate data into the language model; obtaining at least one of a thought process evaluation result for the thought process data and an answer candidate evaluation result for the answer candidate data from the language model; generating an answer to the query based on at least one of the thought process evaluation result and the answer candidate evaluation result through the language model; and providing the generated answer.
[0018] In one embodiment, the answer candidate data may be generated by the language model based on the thought process data.
[0019] In one embodiment, the answer candidate data may be generated by the language model based on the thought process data.
[0020] In one embodiment, the answer may be the result of updating the answer candidate data based on at least one of the thought process evaluation result and the answer candidate evaluation result.
[0021] In one embodiment, the answer candidate data is updated through the language model based on the result of the thought process evaluation, and the answer can be generated through the language model based on the updated answer candidate data and the result of the answer candidate evaluation.
[0022] In one embodiment, in the step of providing the answer, at least one of the thought process evaluation result and the answer candidate evaluation result may be provided together with the answer on at least one service page.
[0023] In one embodiment, if at least one of the thought process evaluation result and the answer candidate evaluation result satisfies a preset condition related to answer quality, the answer is updated through the language model based on at least one of the thought process evaluation result and the answer candidate evaluation result, and the updated answer can be provided on the service page.
[0024] In one embodiment, at least one of the thought process evaluation result and the answer candidate evaluation result can be obtained based on thought basis data extracted from the thought process data by the language model.
[0025] In one embodiment, the accident basis data includes at least one reference data that serves as a criterion for obtaining at least one of the accident process evaluation results and the answer candidate evaluation results, and the reference data may be obtained from at least one external data source.
[0026] In one embodiment, the external data source may include at least one of a search engine and a knowledge database.
[0027] In one embodiment, at least one of the thought process evaluation result and the answer candidate evaluation result is obtained based on at least one state label determined by the language model based on the thought basis data, and the state label may include a plurality of different labels indicating whether the thought basis data matches at least one of the thought process data and the answer candidate data.
[0028] In one embodiment, the different plurality of labels include a first label indicating that the accident basis data matches at least one of the accident process data and the answer candidate data, and a second label indicating that the accident basis data does not match at least one of the accident process data and the answer candidate data; when the state label is the first label, the answer includes the answer candidate data, and when the state label is the second label, the answer may include a result in which the answer candidate data is updated based on at least one of the accident process evaluation result and the answer candidate evaluation result.
[0029] In one embodiment, the thought process evaluation result and the answer candidate evaluation result, each of which may include at least one score calculated by the language model.
[0030] A language model learning method according to the present invention, performed by a computer, may include the steps of: obtaining answer candidate data for at least one learning query; obtaining an evaluation result for the answer candidate data based on the learning query and the answer candidate data; generating learning data based on the evaluation result; and training at least one language model based on the learning data.
[0031] In one embodiment, the evaluation result may be obtained based on the learning query, the answer candidate data, and the thought process data for the learning query.
[0032] In one embodiment, the training data may include answer candidate data in which the evaluation result satisfies a preset condition related to answer quality as preferred sample data, and answer candidate data in which the evaluation result does not satisfy the preset condition as non-preferred sample data.
[0033] In one embodiment, the evaluation result includes at least one state label for the answer candidate data, and the preferred sample data and the non-preferred sample data of the training data can be determined according to the state label.
[0034] In one embodiment, the evaluation result is obtained based on accident basis data extracted from the accident process data, and the accident basis data can be obtained from at least one external data source.
[0035] Meanwhile, the answer providing service system according to the present invention may be configured to include at least one memory storing one or more instructions and at least one processor linked to said at least one memory, and the at least one processor executes said one or more instructions to obtain, from a language model, at least one evaluation result related to an answer to at least one query and the reliability of said answer, and to provide an answer determined based on said evaluation result to at least one service page.
[0036] In one embodiment, the service page may provide at least one of a reliability indicator of the answer and accident basis data regarding the answer together with the answer.
[0037] In one embodiment, if the evaluation result satisfies a preset condition related to the quality of the answer, the answer is provided on the service page, and if the evaluation result does not satisfy the preset condition, an updated result of the answer may be provided on the service page.
[0038] In one embodiment, at least one of the thought process evaluation result for the thought process data and the answer candidate evaluation result for the answer candidate data can be obtained to form a verification path for the answer candidate data through thought basis data extracted from the thought process data, and the number of re-inference operations required for answer generation can be reduced by determining whether to update the answer candidate data through the verification path.
[0039] In one embodiment, the step of generating the training data may include the step of separating the answer candidate data into preferred sample data and non-preferred sample data based on the evaluation result, and the step of training the language model may be configured such that a training signal based on the contrast between the preferred sample data and the non-preferred sample data suppresses the loss of expressive ability acquired in the pre-training stage of the language model.
[0040] In one embodiment, the at least one processor can control that at least one of the operation of obtaining the answer and the operation of obtaining the evaluation result is distributed and executed by an external computing resource linked to the at least one memory through a communication network, and can be configured to determine the answer provided to the service page based on the answer or the evaluation result received from the external computing resource.
[0041] Meanwhile, according to the present invention, a non-transient computer-readable storage medium may be provided for storing instructions that cause the computer to perform an answer generation method when executed by a computer. The answer generation method may include the steps of receiving at least one query, inputting the query into at least one pre-trained language model, obtaining thought process data and answer candidate data for the query from the language model, inputting the thought process data and answer candidate data into the language model, obtaining at least one of a thought process evaluation result for the thought process data and an answer candidate evaluation result for the answer candidate data from the language model, generating an answer to the query based on at least one of the thought process evaluation result and the answer candidate evaluation result through the language model, and providing the generated answer.
[0042] Furthermore, according to the present invention, a non-transient computer-readable storage medium may be provided for storing instructions that cause a computer to perform a language model learning method when executed by a computer. The language model learning method may include the steps of: obtaining answer candidate data for at least one learning query; obtaining an evaluation result for the answer candidate data based on the learning query and the answer candidate data; generating learning data based on the evaluation result; and training at least one language model based on the learning data.
[0043] As described above, according to the present invention, an answer can be generated by simultaneously verifying the thought structure and the answer during the process of generating an answer to a query. Through this, not only the result of the answer generation but also the consistency and reliability of the entire answer generation process can be considered, thereby enabling the generation of a more reliable answer.
[0044] Furthermore, according to the present invention, a multi-stage thought structure and a final answer can be verified and reflected to generate an answer in an Agentic Search Augmentation Generation (Agentic RAG) environment. Through this, consistency throughout the entire multi-stage thought process can be ensured, and accurate answers can be generated even for complex queries based on a consistent thought flow.
[0045] Furthermore, according to the present invention, an answer can be derived by considering the consistency and reliability of the thought structure and results generated throughout the entire operation process of the agent-based AI. Through this, the reliability of the entire execution process can be ensured even in environments involving the use of tools and multi-stage reasoning, and the quality of the answer from the agent-based artificial intelligence system can be improved.
[0046] In addition, according to the present invention, a large language model can be trained to simultaneously verify the thought structure and the answer during the process of generating an answer to a query. Through this, the large language model can be trained to consider the consistency of the generation process rather than simply generating results, thereby improving the ability to generate inference-based responses.
[0047] Specifically, according to the present invention, a training data set in which the thought structure and the answer are simultaneously verified can be generated. Through this, training data reflecting consistency between the generation process and the result can be constructed, and stable inference patterns for various query types can be learned.
[0048] Furthermore, according to the present invention, a large language model can be trained by reflecting the evaluation results of the multi-stage thought structure and the response performed in an agent-based artificial intelligence environment. Through this, the ability to perform consistency-based inference and response generation suitable for an agentic AI environment can be enhanced, and reliability and usability in actual service environments can be improved.
[0049] FIG. 1 illustrates an example of a block diagram of a computing system in which the present invention can be implemented.
[0050] FIG. 2 illustrates an example of a block diagram of a computing device that may be included in a user computing device, a server computing system, and a training computing system, as an embodiment of a computing system in which the present invention can be implemented.
[0051] Figure 3 illustrates an example of a block diagram from another perspective of a computing device, which is one of the components of a computing system.
[0052] FIG. 4 is a conceptual diagram illustrating an answer generation system and a large language model learning system according to the present invention.
[0053] FIG. 5 is a flowchart illustrating a method for generating an answer according to the present invention.
[0054] FIGS. 6a to 6d are conceptual diagrams for explaining the process of generating accident structure data corresponding to a query according to the present invention.
[0055] FIGS. 7a, FIGS. 7b, FIGS. 8a to 8d and FIGS. 9a to 9e are conceptual diagrams for explaining the process of evaluating accident structure data according to the present invention.
[0056] FIGS. 10a to 10d are conceptual diagrams and mathematical formulas for explaining the process of generating accident structure data and performing an evaluation of the accident structure data according to the present invention.
[0057] FIGS. 11a to 11d are conceptual diagrams for explaining a method for generating answers using at least one language model according to the present invention.
[0058] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components are assigned the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not have distinct meanings or roles in themselves. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention.
[0059] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.
[0060] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.
[0061] A singular expression includes a plural expression unless the context clearly indicates otherwise.
[0062] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.
[0063] Terms used in this specification may be defined as follows.
[0064] The terms used in this specification are intended to describe embodiments of the invention and are not intended to limit the scope of the invention. In this specification, singular expressions may include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising,” “having,” and “consisting of” may mean that the features, steps, actions, components, or combinations thereof described in this specification are present, and should be understood not to preclude the existence or addition of one or more other features, steps, actions, components, or combinations thereof.
[0065] Some terms used in this specification may be intended to refer to the same technical meaning of the invention from different perspectives, and the relationship between these terms may be explicitly declared in the definitions below. The definitions below may be intended to ensure that a claim is supported by the corresponding description in the specification when a claim is described using another term representing an equivalent concept, even if that term is used in some paragraphs of this specification.
[0066] In this specification, "query" may refer to request information in a natural language format that is received from a user or an external system and can be processed as input to a large language model or natural language processing engine. "Query" may also be referred to in this specification as "request information," "query information," "input query," "user query," or "natural language query," and all of these expressions are understood to mean the same concept as "query" as used in this specification.
[0067] In this specification, "answer" may refer to output information in a natural language format produced by a large language model or natural language processing engine in response to a query. "Answer" may also be referred to in this specification as "response information," "response expression information," "final answer," "response data," or "answer text," and all of these expressions are understood to mean the same concept as "answer" as used in this specification. The term "answer" is an expression intended to intuitively refer to the output result of this specification from a user's perspective, while the terms "response information" or "response expression information" may be expressions intended to refer to the same output as a data object from a system's perspective.
[0068] In this specification, "Large Language Model (LLM)" may refer to a neural network-based language model that is pre-trained with a large volume of natural language data and capable of outputting natural language corresponding to an input natural language. "Large Language Model" may also be referred to in this specification as "natural language processing engine," "language model engine," "pre-trained language model," "pre-trained inference engine," or "language model," and all of these expressions may be understood to mean the same concept as "Large Language Model" as used in this specification.
[0069] The term "natural language processing engine" above may be used as a term of art to refer to a model as a system configuration unit that can be stored in memory and called by a processor, and is not intended to be unnecessary redundant or indefinite.
[0070] In this specification, "reference large language model" may refer to a pre-trained language model called to generate thought structure data during the training phase. "Reference large language model" may also be referred to in this specification as "reference natural language processing engine," "reference model," "reference model," "judge model," or "reference LLM," and all of these expressions are understood to mean the same concept as "reference large language model" as used in this specification.
[0071] In this specification, "large language model to be trained" may refer to a language model whose parameters are updated during the training phase. "Large language model to be trained" may also be referred to in this specification as "tuning target natural language processing engine," "tuning target model," "target model," "training target engine," or "fine-tuning target model," and all of these expressions are understood to mean the same concept as "large language model to be trained" as used in this specification. The two terms, namely "reference large language model" and "large language model to be trained," may refer to the same model instance or different model instances. According to one embodiment, the two models may share the same prior training weights while only one of the weights is updated during the training phase, or they may be separate models having different prior training weights from the beginning.
[0072] In this specification, "thinking structure data" may refer to the data structure of a multi-stage reasoning result produced by a large language model or natural language processing engine in response to a query, and the multi-stage reasoning result may include at least one of at least one thinking process data and at least one of the answer data to the query. "Thinking structure data" may also be referred to in this specification as "reasoning flow data structure," "reasoning trajectory," "agent trajectory," "thinking flow," "thinking trajectory," or "trajectory," and all of these expressions may be understood to mean the same concept as "thinking structure data" used in this specification.
[0073] In this specification, "thinking process data" refers to an intermediate reasoning unit included in the thinking structure data, and may mean representation information regarding at least one of reasoning, action, or observation performed prior to reaching the answer data. "Thinking process data" may also be referred to in this specification as "intermediate reasoning representation," "reasoning unit," "reasoning data," "reasoning step," "thinking step," or "intermediate reasoning step," and all of these expressions may be understood to mean the same concept as "thinking process data" as used in this specification.
[0074] According to one embodiment, thought process data may be classified into any one of preset types, and said preset types may include at least one of inference types, behavior types, and observation types. The inference type may include expression information regarding internal inference of a large language model, and the behavior type may include expression information regarding external search or external tool call, and
[0075] In this specification, "answer data" refers to response information in natural language format derived through thought process data, as a final output unit included in thought structure data. "Answer data" may also be referred to as "response representation information," "final answer," "response data," or "final answer" in this specification, and all of these expressions may be understood to mean the same concept as "answer data" as used in this specification.
[0076] In this specification, "evaluation data" may refer to an intermediate output calculated based on accident basis data for at least one of accident structure data or accident process data and response data included therein. "Evaluation data" may also be referred to as "verification output data," "verification data," "consistency output data," or "evaluation output" in this specification, and all of these expressions may be understood to mean the same concept as "evaluation data" as used in this specification.
[0077] In this specification, the "evaluation result" refers to information comprehensively calculated regarding accident structure data based on evaluation data, and may include at least one of information indicating the accuracy of the response data and information indicating the evidentiary consistency of the accident process data. The "evaluation result" may also be referred to in this specification as "verification result information," "verification result," "consistency result," or "evaluation verdict," and all of these expressions may be understood to mean the same concept as the "evaluation result" used in this specification.
[0078] In this specification, "step of performing evaluation" or "step in which evaluation is performed" may refer to a series of processing procedures for producing evaluation data defined above and producing evaluation results based thereon. The processing procedures may also be referred to in this specification as "execution of a verification procedure," "consistency determination procedure," or "verification procedure," and all of these expressions may be understood to mean the same concept as "step of performing evaluation" as used in this specification.
[0079] In this specification, "thinking ground data" refers to information in the form of factual units extracted from accident process data classified by observation type among accident structure data, and may mean information that serves as a standard when an evaluation is performed. "Thinking ground data" may also be referred to in this specification as "atomic fact," "evidence piece," "ground object," "ground unit," "unit included in the evidence pool," "fact claim," or "evidence," and all of these expressions are understood to mean the same concept as "thinking ground data" as used in this specification. Thinking ground data may be expressed in the form of a single declarative sentence and may include information that can be directly verified by external documents or search results.
[0080] In this specification, "detailed query data" may refer to information in units of requirements that answer data must satisfy, derived by decomposing from a learning query or request information. "Detailed query data" may also be referred to in this specification as "atomic constraint," "requirement information," "element object," or "atomic constraint," and all of these expressions are understood to mean the same concept as "detailed query data" as used in this specification.
[0081] In this specification, "detailed answer data" may refer to information of a claim unit that can be verified by the cause-of-fact data, derived by breaking down the answer data. "Detailed answer data" may also be referred to as "atomic claim," "claim information," "claim object," or "atomic claim" in this specification, and all of these expressions may be understood to mean the same concept as "detailed answer data" as used in this specification.
[0082] In this specification, "first evaluation indicator" may refer to an indicator regarding the degree of support calculated based on the reasoning data for each inference data included in the thought process data. "First evaluation indicator" may also be referred to in this specification as "reasoning score," "reasoning stage score," "stage evaluation indicator," or "reasoning plausibility score," and all of these expressions may be understood to mean the same concept as "first evaluation indicator" as used in this specification.
[0083] In this specification, "second evaluation indicator" and "third evaluation indicator" may refer to a group of indicators calculated based on the reasoning data for the answer data. The group of indicators may also be referred to in this specification as "final correctness score," "grounded claim score," "constraint score," "answer correctness indicator," or "groundedness indicator," and all of these expressions may be understood to mean the same concept as the second evaluation indicator and the third evaluation indicator used in this specification.
[0084] In this specification, "final evaluation indicator" may refer to an indicator comprehensively calculated for accident structure data by combining at least some of the first evaluation indicator, the second evaluation indicator, and the third evaluation indicator. "Final evaluation indicator" may also be referred to in this specification as "reward score," "reward indicator," "reward score," or "reward signal," and all of these expressions may be understood to mean the same concept as "final evaluation indicator" as used in this specification.
[0085] In this specification, "state label" may refer to a discrete classification marker indicating the relationship between inference data, detailed query data, or detailed answer data, respectively, to accident basis data. "State label" may also be referred to as "state information," "supported state," "supported state," or "support label" in this specification, and all of these expressions may be understood to mean the same concept as "state label" used in this specification. According to one embodiment, the state label may be determined as one of a supported state, a partially supported state, a contradicted state, and an unknown state.
[0086] In this specification, the answer generation system (100) may refer to a computing system configured to include at least one memory and at least one processor, and to call a pre-trained natural language processing engine to produce an answer to a query. The "answer generation system" may also be referred to in this specification as an "answer providing system," "inference system," "response generation system," or "answer generation system," and all of these expressions may be understood to mean the same concept as the "answer generation system" used in this specification.
[0087] In this specification, "large language model learning system" may refer to a computing system comprising at least one memory and at least one processor, configured to acquire thought structure data from a reference natural language processing engine and update the parameters of a natural language processing engine to be tuned. "Large language model learning system" may also be referred to in this specification as "language model learning system," "training system," "parameter update system," or "training system," and all of these expressions may be understood to mean the same concept as "large language model learning system" as used in this specification.
[0088] In this specification, "~part," "~module," and "~unit" included in the answer generation system (100) or the large language model learning system may refer to logical units of instructions stored in memory or instructions executed by a processor, and are not limited to physically separated hardware components. The "~part" may also be referred to as "~processing unit," "~configuration unit," "~management unit," or "~handler" in this specification, and all of these expressions may be understood to mean the same concept as the "~part" used in this specification. According to one embodiment, a plurality of "~parts" may be implemented in a form where they are executed in a time-sharing manner on a single processor, distributed across a plurality of processors and executed in parallel, or executed on a remote server and only the results are transmitted to the answer generation system (100) or the large language model learning system.
[0089] In this specification, "training query" may refer to a query processed as input to a reference large language model or a large language model to be trained during the large language model training phase. "Training query" may also be referred to as "training request information," "training input query," "training query," or "training prompt" in this specification, and all of these expressions may be understood to mean the same concept as "training query" as used in this specification. According to one embodiment, "training query" may be a natural language expression of the same format as a "query" received from a user terminal during the inference phase, and may be distinguished from the "query" of the inference phase in that it is applied during the training phase.
[0090] In this specification, "search query data" refers to information in the form of a query used as input to an external search function or external tool, as expression information of a type of action included in thought process data. "Search query data" may also be referred to as "action expression," "search input," "tool query," or "search input" in this specification, and all of these expressions may be understood to mean the same concept as "search query data" as used in this specification.
[0091] In this specification, "search content" refers to expression information of observation types included in thought process data, which may mean result information received from an external search function or external tool in response to search query data. "Search content" may also be referred to in this specification as "observation expression," "search result information," "retrieval result," or "tool response," and all of these expressions may be understood to mean the same concept as "search content" as used in this specification.
[0092] In this specification, "answer candidate data" is intended to mean the same concept as the answer data defined above, and "answer data" used in this specification may also be expressed as "answer candidate data" when describing claims. That is, "answer candidate data" is an equivalent expression to "answer data" and may refer to response information in natural language format derived through thought process data, as a final output unit included in the thought structure data.
[0093] In this specification, "thought process evaluation result" may refer to the portion derived from the thought process data among the evaluation results. "Thought process evaluation result" may also be referred to as "reasoning evaluation result," "thought stage evaluation result," or "reasoning evaluation result" in this specification, and all of these expressions may be understood to mean the same concept as "thought process evaluation result" as used in this specification. The thought process evaluation result may include information indicating the degree of support, consistency, or reliability of at least one reasoning data included in the thought process data.
[0094] In this specification, "answer candidate evaluation result" may refer to the portion calculated for the answer candidate data (i.e., answer data) among the evaluation results. "Answer candidate evaluation result" may also be referred to as "answer evaluation result," "final response evaluation result," or "answer evaluation result" in this specification, and all of these expressions may be understood to mean the same concept as "answer candidate evaluation result" as used in this specification. The answer candidate evaluation result may include information indicating the accuracy of the answer candidate data, the degree of support for the reasoning basis data, or the suitability of the correct answer.
[0095] In this specification, "reference data" refers to unit information included in the accident basis data, which may mean information serving as the acquisition standard for at least one of the accident process evaluation results and the answer candidate evaluation results. "Reference data" may also be referred to in this specification as "basis unit data," "verification standard information," "fact item," or "reference unit," and all of these expressions are understood to mean the same concept as "reference data" as used in this specification. Reference data may be obtained from at least one external data source.
[0096] In this specification, "external data source" may refer to an external source of information used to obtain reference data. "External data source" may also be referred to as "external information source," "reference data source," or "external information source" in this specification, and all of these expressions may be understood to mean the same concept as "external data source" used in this specification. An external data source may include at least one of a search engine, a knowledge database, a document corpus, a web page, API response data, and an external knowledge base.
[0097] In this specification, a "reliability indicator" is an indicator representing the reliability of a response and may be determined from an evaluation result. The "reliability indicator" may also be referred to in this specification as a "reliability indicator," "confidence indicator," "reliance score," or "confidence indicator," and all of these expressions are understood to mean the same concept as the "reliability indicator" used in this specification. The reliability indicator may be expressed in at least one of a score, a rating, a color display, an icon, or a text message.
[0098] In this specification, "preferred sample data" may refer to candidate answer data included in the training data that the model should follow, and "non-preferred sample data" may refer to candidate answer data included in the training data that the model should avoid. "Preferred sample data" may also be referred to as "positive sample," "positive sample," "preferred sample," or "chosen sample" in this specification, and "non-preferred sample data" may also be referred to as "negative sample," "negative sample," "non-preferred sample," or "rejected sample." All of these expressions are understood to mean the same concept as "preferred sample data" and "non-preferred sample data" as used in this specification.
[0099] Additionally, in the step of generating training data disclosed in this specification, a multi-criteria filtering step may be performed to classify answer candidate data into preferred sample data or non-preferred sample data based on evaluation results. According to one embodiment, the multi-criteria filtering step may apply at least one of (i) a first filtering criterion based on the result of evaluating the thought process, (ii) a second filtering criterion based on the result of evaluating the answer candidate, and (iii) a third filtering criterion based on whether it matches the reasoning data. The classified preferred sample data and non-preferred sample data may be used to calculate a loss function in the learning step, and the loss function may be configured to update model parameters in a direction that increases the likelihood of the model output for preferred sample data and in a direction that decreases the likelihood of the model output for non-preferred sample data.
[0100] In this specification, "first label" and "second label" are different labels included in the status label and may refer to markers for discretely classifying whether the accident basis data matches at least one of the accident process data and the answer candidate data. The first label is a label indicating a match, and the second label is a label indicating a discrepancy. The first label may also be referred to in this specification as "supported," "match label," or "positive label," and the second label may also be referred to as "contradicted," "discord label," or "negative label," and all of these expressions may be understood to mean the same concept as "first label" and "second label" used in this specification, respectively.
[0101] In this specification, the "combined evaluation indicator" is an indicator calculated by integrating the results of the thought process evaluation and the results of the answer candidate evaluation, and can be input into the step of generating an answer. The "combined evaluation indicator" may also be referred to as the "integrated evaluation indicator," "final evaluation indicator," or "combined evaluation indicator" in this specification, and all of these expressions should be understood to mean the same concept as the "combined evaluation indicator" used in this specification.
[0102] In this specification, "service page" may mean an interface or display area for providing a response to a user. "Service page" may also be referred to in this specification as "user interface page," "response display page," "user interface page," or "response display page," and all of these expressions may be understood to mean the same concept as "service page" as used in this specification. A service page may be implemented in at least one form among a web page, a mobile application screen, a desktop application screen, or a chatbot interface displayed on the screen of a user terminal.
[0103] In this specification, "training data set" may refer to a set of data configured for updating parameters of a large language model to be trained. According to one embodiment, the training data set may include at least one training query, thought structure data corresponding to the training query, and at least some of the evaluation results for the thought structure data. The "training data set" may also be referred to in this specification as "training data set," "training corpus," "fine-tuning dataset," or "training corpus," and all of these expressions may be understood to mean the same concept as the "training data set" used in this specification.
[0104] In addition, the learning query, thought process data, and answer candidate data disclosed in this specification may have a triple dependency structure during the process of generating the learning data set. According to one embodiment, the learning query serves as a criterion for generating the thought process data, and the learning query and thought process data serve as criteria for generating the answer candidate data, and the learning query, thought process data, and answer candidate data can all be utilized as criteria for calculating evaluation results. Such a triple dependency structure can provide the effect of improving data flow consistency between the learning stage and the inference stage when compared to a conventional learning data set in which the learning data set includes only evaluation results for the answer candidate data.
[0105] Hereinafter, the present invention will be examined in more detail with reference to the attached drawings. FIG. 1 illustrates an example of a block diagram of a computing system in which the present invention can be implemented. FIG. 2 illustrates an example of a block diagram of a computing device that may be included in a user computing device, a server computing system, and a training computing system as an embodiment of a computing system in which the present invention can be implemented. FIG. 3 illustrates an example of a block diagram of a computing device in another aspect that is one of the components of a computing system.
[0106] Meanwhile, FIG. 1 illustrates an example of a block diagram of a computing system in which the present invention can be implemented. In this regard, the answer generation system and the large language model learning system according to the present invention can be implemented through the computing device described below and can perform data processing related to the aforementioned answer generation method and large language model learning method (or language model learning method).
[0107] In the present invention, the terms “language model learning method and system” and “large language model learning method and system” may be used as identical or substantially corresponding concepts, and for convenience of explanation, they may be used interchangeably below. For example, the language model in the present invention may include a Large Language Model (LLM) trained based on large-scale language data, and accordingly, the language model learning method and system may be interpreted as having the same meaning as the large language model learning method and system.
[0108] Referring to FIG. 1, a computing system (10000) for performing the answer generation method and the large language model learning method of the present invention may include at least one computing device. At this time, the at least one computing device may be a single processor or a multiprocessor computing device.
[0109] The components of at least one computing device of the present invention may include at least one processor, memory, other hardware, and various hardware components such as a system bus (not shown) that connects various system components to transmit and receive data to and from each other (e.g., telecommunicationally connected, physically connected, electrically connected), and the components of at least one computing device are not limited thereto and may be very diverse.
[0110] Meanwhile, at least one computing device included in the computing system (10000) that performs the answer generation method and the large language model training method may be connected to communicate via a network (1070). For example, at least one computing device included in the computing system (10000) may be clustered or may be part of a local area network (LAN). Additionally, at least one computing device may be part of a wide area network (WAN) or connected to at least one of a client-server network and a peer-to-peer network within the cloud.
[0111] Meanwhile, when at least one computing device is used in at least one of a network environment and a cloud computing environment, the at least one computing device may be connected to at least one of a public and a private network through a network interface or an adapter. In one embodiment, other communication connection devices, such as a modem, may be used to establish communication through the network. The modem may be at least one of an internal modem and an external modem, and may be connected to a system bus through a network interface or a specific mechanism, etc. A wireless network component consisting of an interface and an antenna may be coupled to the network through a device such as an access point, a peer computer, etc. In the present invention, the method by which at least one computing device is connected to communicate through the network (1070) is not limited and may be connected to communicate in a manner different from the described examples.
[0112] Furthermore, other computer-type devices or systems not shown in FIG. 1 may also interact technically with at least one computing device or other system through one or more connections to the network (1070) via a network interface. Here, the network interface may include network interface equipment such as a physical network interface controller (NIC) or a virtual network interface (VIF).
[0113] The network (1070) of the present invention may include various forms such as the Internet, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth (Bluetooth™), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wireless USB (Wireless Universal Serial Bus), etc., and in the present invention, data transmission may be performed based on standard communication protocols such as TCP / IP, HTTP, SSL, etc.
[0114] A computing system (10000) for performing an answer generation method and a large language model learning method according to the present invention may include at least one of a user computing device (1010), a training computing system (1050), and a server computing system (1030).
[0115] A user computing device (1010) according to the present invention may be understood as a computing device comprising at least one processor (1011) and memory (1012) for performing an answer generation method and a large language model learning method. For example, the user computing device (1010) may include at least one computing device among a smartphone, a smart TV, a laptop computer, a desktop computer, a digital broadcasting terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, a slate PC, a tablet PC, an ultrabook, a wearable device (e.g., a smartwatch, a smart glass, and a head-mounted display).
[0116] At least one processor (1011) constituting the user computing device (1010) may include one or more general-purpose processors or one or more special-purpose processors. For example, at least one processor (1011) constituting the user computing device (1010) may be composed of at least one of a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), an arithmetic logic unit (ALU), a floating-point arithmetic unit (FPU), an application integrated circuit, an application semiconductor (ASIC), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electrical units for performing functions, or a plurality of electrically connected processors.
[0117] Furthermore, at least one processor (1011) may be configured to execute a computer-readable instruction contained in memory (1012) or other instructions described herein.
[0118] The memory (1012) constituting the user computing device (1010) according to the present invention may include volatile memory, non-volatile memory, fixed media, removable media, magnetic media, optical media, semiconductor media, or other types of physically durable storage media.
[0119] For example, memory (1012) may include one or more non-transient or transient computer-readable storage media such as RAM, ROM, HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), EEPROM, EPROM, flash memory device, magnetic disk, etc., and may include web storage of a server that performs the storage function of memory on the internet. Such memory (1012) may store data and instructions necessary for the at least one processor (1011) to train (or optimize, fine-tune) a large language model and to perform the operation of an application to provide an answer to a query using the pre-trained large language model.
[0120] A user computing device (1010) may include one or more user input components (1021) that detect user input. For example, the user input component (1021) may also be referred to as a user interface module. The user input component (1021) may include a touch screen, a computer mouse, a keyboard, a keypad, a touchpad, a trackball, a joystick, a voice recognition module, or other similar devices. However, the present invention does not limit the type of user input component (1021). In this case, the user input component (1021) in the present invention does not necessarily mean a hardware means, but can be understood as a channel for receiving input from a user. Meanwhile, the user of the present invention may refer to an automated agent, script, playback software, etc., that operates on behalf of one or more people.
[0121] A user can interact with a computing system (10000) including at least one computing device through input text, touch, voice, movement, computer vision, gesture, or other forms of input or output using a user input component (1021). For example, the user input component (1021) may include one or more of a command line interface (CLI), a graphical user interface (GUI), a natural user interface (NUI), a voice command interface, or other user interface (UI) representations.
[0122] Between the user input component (1021) and the user computing device (1010), one or more application programming interface (API) calls may be made based on user input received from a user interface or network.
[0123] Here, the expression "based on" may be interpreted to include cases where it is based on the use of a specific configuration, modified from, derived from, influenced by, dependent on, or otherwise derived from a specific configuration. In some embodiments, an API call may be configured for a specific API, which may be interpreted or converted into an API call configured for another API. Here, an API may refer to a defined interface or connection between computers or between computer programs.
[0124] In one embodiment, the user computing device (1010) may store at least one machine learning model (1020). For example, the user computing device (1010) may be various machine learning models, such as a plurality of neural networks (e.g., deep neural networks) that perform at least one of generating answers to queries and learning large language models, or other types of machine learning models including non-linear models or linear models, and may be composed of a combination thereof.
[0125] According to an embodiment of the present invention, a user computing device (1010) may perform an answer generation method and a large language model learning method using a local or external machine learning model (1020). Alternatively, the user computing device (1010) may perform an answer generation method and a large language model learning method using a machine learning model (1040) provided by a server.
[0126] In addition, according to another embodiment of the present invention, a server computing system (1030) communicating with a user computing device (1010) may provide an answer to a query according to the present invention to the user computing device (1010) via an application or the web in accordance with a request from a user received through the user computing device (1010).
[0127] In addition, according to another embodiment of the present invention, by performing an answer generation method and a large language model learning method of a user computing device (1010) and a server computing system (1030), an answer to a query generated using a pre-trained large language model can be provided to the user.
[0128] Additionally, according to various embodiments of the present invention, a user computing device (1010) or a server computing system (1030) can learn machine learning models (1020, 1040) performed in an answer generation method and a large language model learning method through interaction with a training computing system (1050) that is communicatedly connected via a network (1070). In this case, the training computing system (1050) may be a computing system separate from the server computing system (1030). Alternatively, in some embodiments, the training computing system (1050) may be part of the server computing system (1030) or part of the user computing device (1010).
[0129] Meanwhile, the server computing system (1030) may include at least one processor (1031) and memory (1032). Here, the processor (1031) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), an application integrated circuit, an application semiconductor (ASIC), an arithmetic logic unit (ALU), a floating-point arithmetic unit (FPU), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electrical units for performing functions. For example, at least one processor (1031) may include a circuit and a transistor configured to execute instructions from memory (1032).
[0130] The memory (1032) constituting the server computing system (1030) according to the present invention may include volatile memory, non-volatile memory, fixed media, removable media, magnetic media, optical media, semiconductor media, or other types of physically durable storage media. For example, the memory (1032) may include one or more non-transient or transient computer-readable storage media such as RAM, ROM, HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), EEPROM, EPROM, flash memory device, magnetic disk, etc., and combinations thereof, and may include web storage of a server that performs the storage function of memory over the internet. Additionally, the server computing system (1030) may further include a data storage (data store). For example, the data storage may be composed of at least one of a relational database, a NoSQL database, a data warehouse, and a local file system.
[0131] In the memory (1032) constituting the server computing system (1030) according to the present invention, data and instructions necessary for the at least one processor (1031) to perform the operation of an application for at least one of generating an answer to a query and learning a large language model may be stored.
[0132] In one embodiment, the server computing system (1030) may be composed of a single device or a plurality of computing devices, and may be configured to operate according to a sequential architecture or a parallel computing architecture. Additionally, a distributed processing system may be configured with a plurality of networked devices.
[0133] Meanwhile, the training computing system (1050) may include at least one processor (1051) and memory (1052). The model trainer (1060) is a logical component that executes the training of at least one machine learning model (1020, 1040) and may be implemented in the form of hardware, firmware, or software. For example, the model trainer (1060) may be executed by the processor (1051) after loading training data (1061) stored in a storage device into memory (1052). For example, the model trainer (1060) may be configured to execute one or more operations (e.g., model training, model reconstruction, model validation, model testing) on at least one machine learning model.
[0134] The machine learning model of the present invention may include at least one of a statistical model, an algorithm, a neural network (NN), a convolutional neural network (CNN), a generative neural network, a Word2Vec model, a Bag of Words model, a TF-IDF (document frequency-inverse document frequency) model, a GPT (Generative Pre-trained Transformer) model (or other autoregressive models), a PPO (Proximal Policy Optimization) model, a nearest neighbor model (Ex: k-nearest neighbor model), a linear regression model, a K-means clustering model, a Q-learning model, a TD (Temporal Difference) model, a Deep Adversarial Network model, and all other types of models further described herein.
[0135] Specifically, the model trainer (1060) may execute operations to train a machine learning model, and said operations may include at least one of adding, removing, and modifying model parameters. At this time, the training of the machine learning model may be at least one of supervised learning, semi-supervised learning, and unsupervised learning. In one embodiment, the training of the machine learning model may include the step of repeatedly inputting training data (1061) based on epochs and repeatedly performing the machine learning model training process configured in this way. Here, an epoch may refer to a unit in which the entire set of training data (1061) undergoes forward and backpropagation processing once. In some implementations, different levels of training methods (e.g., supervised learning, semi-supervised learning, unsupervised learning) may be used for different epochs.
[0136] The training data (1061) of the present invention may include input data or data previously output from at least one machine learning model (e.g., recursive learning feedback).
[0137] At least one parameter of a machine learning model may include at least one of a seed value, a model node, a model layer, an algorithm, a function, connections between different machine learning models, connections between parameters, machine learning model constraints, and other digital components that influence the output of the machine learning model. In this case, model connections between different machine learning models may include or represent model parameters or relationships between models, and these may be dependent or interdependent, hierarchical, static, or dynamic. The combinations and configurations of model parameters described herein may be too complex to be maintained or utilized by human cognitive abilities.
[0138] In the present invention, the machine learning parameters described according to the embodiments are not limited, and a single machine learning model may further include a plurality of model parameters.
[0139] Meanwhile, FIG. 2 illustrates an example of a block diagram of a computing device (1100) that may be included in a user computing device (1010), a server computing system (1030), and a training computing system (1050), as an embodiment of a computing system (10000) in which the present invention can be implemented.
[0140] As illustrated in FIG. 2, the computing device (1100) may include at least one application (e.g., Application 1 to Application N), and each of the at least one application may include a machine learning library and a model execution environment for performing a machine learning-based answer generation method and a large language model training method. The at least one application included in the computing device (1100) may communicate with the sensor, context manager, device state manager, or additional component(s) within the computing device (1100) via an Application Programming Interface (API). In one embodiment, the at least one application may interface with device components, such as receiving sensor data or state data or transmitting prediction results to an output device via a public or private API.
[0141] Meanwhile, FIG. 3 illustrates an example of a block diagram in another aspect of a computing device (1200), which is one of the components of a computing system (10000) that performs an answer generation method and a large language model learning method according to an embodiment of the present invention.
[0142] A computing device (1200) according to the present invention may include at least one application (e.g., Application 1 to Application N), and at least one application may communicate with a central intelligence layer (1210). Each application may interact with a shared model within the central intelligence layer (1210) through an API (e.g., a common API).
[0143] The central intelligence layer (1210) includes one or more machine learning models and may share them among multiple applications or provide them independently to each. In one embodiment, the central intelligence layer (1210) may be integrated as part of an operating system or implemented as a separate logical layer.
[0144] Additionally, the central intelligence layer (1210) can communicate with the central device data layer (1220). The central device data layer (1220) can integrate and store a training data set stored within the computing device (1200) and provide it as input data necessary for training (e.g., fine-tuning) a large language model to be trained. Each device component (e.g., sensor, state manager, etc.) can communicate with the central device data layer (1220) via a private API, etc.
[0145] The technology described herein may be composed of a single or multiple computing devices, and the machine learning model performing the answer generation method and the large language model training method may be executed sequentially or in parallel on one component or multiple distributed components. The data storage, machine learning model, and application may be distributed and operated locally or over a network, and these configurations can be flexibly applied to various system architectures.
[0146] The above describes a computing system in which the answer generation method and the large language model training method according to the present invention can be implemented. Below, the answer generation method and the large language model training method according to the present invention will be described in general.
[0147] The answer generation method and large language model learning method disclosed in this specification can produce evaluations of answer data and evaluations of thought process data as separate evaluation data, unlike conventional answer generation evaluation methods which could judge only the correctness of the final answer as a single criterion. Specifically, the evaluation procedure disclosed in this specification can produce evaluation data based on the consistency of at least one detailed answer data corresponding to the answer data (e.g., second evaluation data or third evaluation data) and evaluation data based on the consistency of at least one inference data included in the thought process data (e.g., first evaluation data) through separate procedures. Accordingly, the evaluation procedure disclosed in this specification can identify the consistency of the answer data and the consistency of the thought process data as one of four consistency combinations. For example, the four consistency combinations may include a first combination in which the answer data is correct and the thought process data is supported by external information, a second combination in which the answer data is correct but the thought process data is not supported by external information, a third combination in which the answer data is incorrect but the thought process data is supported by external information, and a fourth combination in which the answer data is incorrect and the thought process data is not supported by external information. Among these four consistency combinations, cases corresponding to the second and third combinations may correspond to cases that are not identified by the conventional final answer single criterion evaluation.
[0148] The answer generation method and large language model learning method disclosed herein can specify a first state label for at least one inference data based on thought basis data, and can specify a fourth state label for at least one detailed answer data based on thought basis data. In this case, a first evaluation indicator calculated based on the first state label and a second evaluation indicator or a third evaluation indicator calculated based on the fourth state label can be configured to calculate the consistency of the thought process data and the consistency of the answer data as separate indicators. For example, if the first evaluation indicator is above a threshold and the second evaluation indicator or the third evaluation indicator is above the threshold, it may correspond to the first combination, and if the first evaluation indicator is below the threshold and the second evaluation indicator or the third evaluation indicator is above the threshold, it may correspond to the second combination. By calculating such separate indicators, the training data set generation procedure disclosed herein can exclude thought structure data from the training data set where the thought process data is not supported by external information, even if the answer data corresponds to the correct answer. In addition, the training data set generation procedure disclosed in this specification can select thought structure data as a training target that satisfies both threshold conditions regarding the consistency of answer data and the consistency of thought process data.
[0149] The effect of such a separated evaluation procedure may be reflected in the claims by the limitation of Claim 1 disclosed herein, “at least one large language model pre-trained to generate an answer to the query based on at least a portion of the thought structure data and the evaluation results for said thought structure data,” and the limitation of Claim 7, “the step of obtaining evaluation data for at least one of the thought process data included in said thought structure data and the answer data for said learning query.” Additionally, the first to fourth state labels, the first to third evaluation data, and the first to third evaluation indicators described in Claims 9 to 14 may correspond to the limitation for calculating the consistency of the answer data and the consistency of the thought process data as separate data objects. By the separated evaluation procedure disclosed herein, a procedure for identifying cases in which the thought process data is not supported by external information, even if the answer data corresponds to the correct answer. Furthermore, the separated evaluation procedure disclosed herein may provide a procedure for separately applying conditions regarding the consistency of answer data and conditions regarding the consistency of thought process data in the configuration of a training data set. Moreover, the separated evaluation procedure disclosed herein may provide a learning procedure for updating the parameters of a large language model by combining a loss function for the consistency of answer data and a loss function for the consistency of thought process data. Such a separated evaluation procedure may correspond to a data processing procedure different from the conventional single-criterion evaluation method for final answers, and may lead to an environment in which consistency indicators for answer data and consistency indicators for thought process data are calculated together.
[0150] In this specification, "confidence score" may refer to a numerical value representing the degree of confidence in an answer, calculated based on the final evaluation indicator included in the evaluation results of the thought structure data. "Confidence score" may also be referred to in this specification as "confidence indicator," "answer confidence," "reward conversion score," or "answer confidence," and all of these expressions may be understood to mean the same concept as "confidence score" as used in this specification. According to one embodiment, the confidence score may be expressed in at least one form among a numerical value, a grade, a percentage, or a visual indicator converted according to a preset scale.
[0151] The present invention relates to a method and system for generating answers to a query using at least one artificial intelligence model. Specifically, the present invention can generate answers to a query using Agentic Retrieval-Augmented Generation (Agentic RAG) technology. Here, an agent may refer to an artificial intelligence entity that independently makes decisions to perform multiple tasks to resolve a given query, and can autonomously perform a series of tasks such as generating search query data according to the query, calling external tools, or performing intermediate inference.
[0152] Furthermore, Retrieval-Augmented Generation (RAG) can refer to a technology that retrieves relevant information from external documents or databases and generates answers based on the retrieved information. Such agent-based RAG technology can go beyond a single-stage search and generation process to form a multi-stage thought structure that includes multiple stages of search, information acquisition, and inference to resolve a query, and can derive answers step-by-step based on intermediate information and results generated at each thought stage.
[0153] Specifically, in the present invention, the operation process of an agent can be implemented as a series of answer generation processes performed by a large language model. Accordingly, in the specification according to the present invention, the operation of the agent can also be interpreted as the operation of the large language model, and the answer generation process performed by the large language model can be interpreted as the operation process of the agent.
[0154] More specifically, the present invention relates to an answer generation method and system capable of generating an answer to a query by utilizing a pre-trained large language model to verify the thought structure and the answer together during the process of generating an answer to a query. Herein, according to the present invention
[0155] The thought structure according to the present invention may refer to a flow of multiple stages of thought processes formed in the process of generating an answer to a query. In this case, the thought structure according to the present invention may be named a sequential reasoning flow, a reasoning trajectory, or a thought flow.
[0156] Specifically, in the present invention, thought structure data corresponding to a query can be generated during the answer generation process, and said thought structure data may include at least one of a query, thought process data corresponding to the query, and answer data for the query. At this time, the thought process data may include at least one of search query data generated in response to the query and search content obtained through the search. For example, the thought process data may be configured to include various data derived from multiple stages, such as a thinking stage, an information search stage, and an information utilization stage (observation or information), generated for the resolution of the query.
[0157] The present invention can generate an answer to a query in such an agent-based RAG environment by utilizing at least one large language model that has been pre-trained to generate an answer to the query based on at least some of the thought structure data corresponding to the query and the evaluation results of the thought structure data. In this case, the present invention can evaluate the multi-stage thought structure formed during the answer generation process and the final answer together to produce the thought process during the answer generation process and the evaluation results for the answer together.
[0158] Meanwhile, the present invention relates to a large language model training method and system capable of training a large language model to generate an answer by simultaneously verifying the thought structure and the answer during the process of generating an answer to a query.
[0159] In the present invention, a large language model can be trained to generate an answer to a query based on at least some of the thought structure data corresponding to the query and the evaluation results of the thought structure data. At this time, "training a large language model" may refer to a process of adjusting the training parameters of a large language model by retraining, fine-tuning, or additionally training some or all of the parameters of the large language model using a given training data set, so that the pre-trained large language model evaluates both the thought structure and the answer during the answer generation process, and generates an answer to the query based on the evaluation results.
[0160] Specifically, the present invention can construct (or generate, configure) a training dataset for training a large language model to generate an answer to a query based on at least some of the thought structure data corresponding to the query and the evaluation results of the thought structure data. Furthermore, the present invention can train a large language model to generate an answer to a query according to a stepwise inference flow using the generated training dataset.
[0161] Meanwhile, the answer generation system and the large language model learning system according to the present invention may include at least one artificial intelligence model. In this case, the at least one artificial intelligence model mentioned in the present invention may include various artificial intelligence models that can be utilized depending on various situations or purposes. For example, the artificial intelligence model of the present invention may include at least one of a Large Language Model (LLM), a super-large Foundation model, a Generative AI model, a Vision Language Model (VLM), a Machine Learning (ML) model, a Deep Learning model, a Deep Neural Network (DNN), a Transformer-based model, a Supervised Learning (SL) model, a Reinforcement Learning (RL) model, and a special purpose model (e.g., a time series forecasting model (e.g., ARIMA model, SARIMA model, etc.), a time series Foundation model, a Graph Neural Network, a Multimodal Model, a Natural Language Processing (NLP) model, a computer vision model, a speech recognition model, a synthesis model, a recommendation system model, etc.).
[0162] Furthermore, the artificial intelligence model used in the answer generation system and large language model learning system according to the present invention may be implemented with a single or multiple components. More specifically, the artificial intelligence model used in the present invention may be implemented with a single component (1) or with multiple components (2, 3, 4 or more, etc.).
[0163] In this regard, when multiple artificial intelligence models are implemented in the present invention, the multiple artificial intelligence models may be implemented as models of the same type. In this case, the multiple artificial intelligence models may be implemented to perform different functions (or roles) in the present invention.
[0164] In the foregoing, the answer generation method and the large language model learning method according to the present invention have been generally described, and these can be implemented by the answer generation system and the large language model learning system described below. Below, with reference to FIG. 4, the answer generation system and the large language model learning system according to the present invention will be described in detail. FIG. 4 is a conceptual diagram for explaining the answer generation system and the large language model learning system according to the present invention.
[0165] Furthermore, the answer generation method according to the present invention can be implemented through a self-checking answer generation mechanism. Specifically, at least one language model may first generate thought process data and answer candidate data for at least one query, and then the same language model or a separate language model may perform an evaluation of the thought process data and answer candidate data to produce at least one of the thought process evaluation result and the answer candidate evaluation result, and the self-checking may be performed in a manner that generates an answer based on the produced evaluation result. This self-checking method allows for the consideration of consistency and reliability throughout the entire answer generation process, not just the answer generation result, and enables the generation of a more reliable answer.
[0166] In addition, the answer generation method according to the present invention can be implemented through a separate evaluation method that separates and calculates the result of the thought process evaluation and the result of the answer candidate evaluation. Specifically, the result of the thought process evaluation may include information indicating the degree of support, consistency, or reliability of at least one inference data included in the thought process data as an evaluation result of the thought process data, and the result of the answer candidate evaluation may include information indicating the accuracy of the answer candidate data, the degree of support for the thought basis data, or the suitability of the correct answer as an evaluation result of the answer candidate data. Through this separate evaluation method, evaluations can be performed independently for the thought process data and the answer candidate data, respectively, and an answer can be generated based on at least one of the respective evaluation results.
[0167] Meanwhile, the answer candidate data according to the present invention can be generated by a language model based on thought process data. That is, after receiving a query as input, the language model can primarily generate thought process data for the query and generate answer candidate data based on the generated thought process data. By generating answer candidate data based on thought process data in this way, the answer candidate data can be formed consistently with the thought process data, and the consistency between the thought process and the answer candidate can be effectively verified in a subsequent evaluation step.
[0168] In addition, the answer according to the present invention may be generated as a result of updating answer candidate data based on at least one of the result of evaluating the thought process and the result of evaluating the answer candidate. Specifically, the answer candidate data may be updated first through a language model based on the result of evaluating the thought process, and an answer may be generated through a language model based on the updated answer candidate data and the result of evaluating the answer candidate. This sequential update flow enables more precise answer updating by ensuring that the two evaluation results are processed in separate steps rather than in a single call and reflected in the generation of the answer.
[0169] Meanwhile, each of the thought process evaluation result and the answer candidate evaluation result according to the present invention may include at least one score calculated by a language model. The score may be expressed in a quantitative form, for example, as a real number between 0 and 1, an integer greater than or equal to 0, or an indicator value based on an arbitrary scale. The thought process evaluation result may include at least one score for thought process data, and the answer candidate evaluation result may include at least one score for answer candidate data. Evaluation results in this score form may be utilized as quantitative input in the answer generation stage and may be used for subsequent branch processing or condition determination.
[0170] The results of the accident process evaluation and the candidate answers produced in this manner are not limited to being expressed as simple text output, but can be converted into structured data including at least one status label, at least one numerical score, and at least one reference information regarding the accident basis data, and utilized in subsequent steps. That is, the evaluation results disclosed in this specification may have an explicit data structure that can be observed and processed outside the model responsible for answer generation, and thereby may have a verifiable processing structure that is distinguished from black-box processing, which is unverifiable from the outside, in the answer generation process.
[0171] Meanwhile, as illustrated in FIG. 4, the answer generation system (100) according to the present invention may include at least one of a communication unit (110), a storage unit (120), a control unit (130), and a learning unit (140). However, the components of the answer generation system (100) according to the present invention are not limited thereto and may further include various hardware components that perform the same or similar roles as described in the description of the present specification.
[0172] The answer generation system (100) according to the present invention may also be named an answer provision service system, and for convenience of explanation, the answer generation system (100) and the answer provision service system (100) may be used interchangeably with the same meaning below.
[0173] Although not illustrated, the answer generation system (100) and language model learning system (hereinafter referred to as the “learning system” and described, not illustrated) according to the present invention may include at least one processor, and such processor may include one or more general-purpose processors or one or more special-purpose processors (e.g., digital signal processor, tensor processing unit (TPU), graphics processing unit (GPU), neural network processing unit (NPU), application integrated circuit, application semiconductor (ASIC), field programmable gate array (FPGA), quantum processing unit (or quantum processor, QPU), etc.). At least one processor may be configured to execute instructions, computer-readable instructions, or other instructions described herein that are stored (or included) in the storage unit (120).
[0174] In the answer generation system (100) and large language model learning system according to the present invention, a memory and at least one processor can cooperate to perform the data processing described below. The processor can perform a series of operations and data processing using data and information stored in the memory. At this time, the memory may be a component of the storage unit (120).
[0175] In addition, the answer generation system (100) and the learning system according to the present invention can perform data processing and computation processes using quantum gates, quantum entanglement, and quantum superposition states, taking into consideration implementation in a quantum computer environment. For example, the present invention can perform parallel computations based on qubits, and such quantum computations can operate complementarily with existing classical computers.
[0176] Such quantum computers may include parallel computation using qubits and high-speed data processing devices utilizing quantum entanglement, and hardware-based computational acceleration using FPGAs and ASICs is possible. In addition, quantum computers may utilize quantum processors capable of qubit-based parallel computation and can increase data throughput through a hybrid structure with existing classical computers.
[0177] Meanwhile, the answer generation system (100) and the learning system according to the present invention may exist inside a server (hereinafter referred to as the server) built to perform a specific purpose (e.g., generating answers to queries and learning large language models), or they may exist as a separate system from said server. When the answer generation system (100) and the learning system exist inside the server, various services related to the present invention (e.g., providing answers to queries, generating learning data sets and learning large language models, etc.) may be provided through at least one component located inside the server, or through configuration modules that perform functions similar to said components. In this case, the application may provide various services related to the present invention on an electronic device (or user terminal (10)) on which the application is installed, through communication with the server.
[0178] Meanwhile, the answer generation system (100) according to the present invention may be configured to include a learning system (not shown). In this case, the learning system may provide various services related to the present invention (e.g., generation of a learning data set and training of a large language model) through the communication unit (110), storage unit (120), control unit (130), and learning unit (140) of the answer generation system (100). As an example, the learning unit (140) of the answer generation system (100) may refer to the learning system.
[0179] In the present invention, the answer generation system (100) is described as including a learning system (or learning unit (140)) that performs learning on a large language model (131) to generate an answer to the query based on at least some of the thought structure data and the evaluation results of the thought structure data, but is not limited thereto. The learning system (or learning unit (140)) may be linked with an external server or a cloud computing environment to generate a learning data set for learning the large language model and perform a large language model learning method. At this time, the large language model (131) learned by the large language model learning method according to the present invention may be used in the answer generation system (100).
[0180] That is, the answer generation system (100) according to the present invention may include a learning system, or the learning system may exist as a separate and independent system from the answer generation system (100). For convenience of explanation, the answer generation system (100) is described below as including a learning system, and the learning system performs a large language model learning method through at least one of the communication unit (110), storage unit (120), control unit (130), and learning unit (140) of the answer generation system (100). Additionally, the answer generation system (100) and the learning system may be used in combination, and in this case, the answer generation system (100) may be described as performing the function of the learning system.
[0181] Meanwhile, the communication unit (110) according to the present invention may be connected via a wireless or wired network to a user terminal (10), an LLM server (150), a DB server, a cloud server, an external server, a central server, a device, and at least one network, etc., to receive or transmit overall data and information necessary for the operation of the answer generation system (100) according to the present invention.
[0182] The communication unit (110) can receive a user query (or, query) from the user terminal (10). Here, “receiving a user query (or, query)” may mean receiving an input signal corresponding to the user query based on the user query being input by the user through the input unit configuration provided in the user terminal (10).
[0183] Here, the user terminal (10) may include at least one of a mobile phone, a smartphone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, a wearable device (e.g., a smartwatch, a smart glass, a head-mounted display).
[0184] In addition, the input section of the user terminal (10) in the present invention does not necessarily refer to a hardware means, but can be understood as a channel for receiving input from a user. Such an input section of the user terminal (10) may also be referred to as a user interface module. The input section of the user terminal (10) may include a touch screen, a computer mouse, a keyboard, a keypad, a touchpad, a trackball, a joystick, a voice recognition module, or other similar devices. However, the present invention does not limit the type of input section of the user terminal (10). Here, user input may include documents, text, images (or videos), voice, etc. In this case, the answer generation system (100) may further include a module for converting voice into text.
[0185] Meanwhile, the communication unit (110) may include at least one communication module capable of wireless communication and wired communication between the answer generation system (100) and at least one communication target. Additionally, the communication unit (110) may include a communication module that connects the answer generation system (100) to at least one network.
[0186] The communication unit (110) communicates with a database (200), an LLM server (150), and at least one external server (300) (e.g., a web server, a document storage server, a knowledge database server, a search engine server, or an API-based information provision server, etc.) to receive data necessary to perform the answer generation method and the large language model learning method according to the present invention.
[0187] Meanwhile, the communication unit (110) can support various communication methods depending on the communication standard of the device being communicated. For example, the communication unit (110) may be configured to communicate with a communication target using at least one of the following technologies: WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth (Bluetooth™), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), and Wireless USB (Wireless Universal Serial Bus).
[0188] Next, the storage unit (120, or memory) serves to store various data related to the present invention, and the storage unit (120) may be provided in the answer generation system (100) itself, or alternatively, at least part of the storage unit (120) may mean a database (DB) or memory. The storage unit (120) may include one or more non-transient computer-readable storage media that can be read or accessed by at least one processor. One or more computer-readable storage media may include volatile or non-volatile storage components such as optical, magnetic, organic, or other memory or disk storage devices. In some examples, the storage unit (120) may be implemented using a single physical device (e.g., one optical, magnetic, organic, or other memory or disk storage device), whereas in other examples, the storage unit (120) may be implemented using multiple physical devices.
[0189] The storage unit (120) may include computer-readable instructions and additional data. The storage unit (120) may include a storage necessary to perform at least some of the methods, scenarios, and techniques described in this specification, or at least some of the functions of the device and network. Furthermore, at least some of the storage unit (120) may be a cloud storage or a cloud server. That is, the storage unit (120) is sufficient as a space where information necessary for the operation of the answer generation system (100) according to the present invention is stored, and it can be understood that there are no restrictions on the physical space. Accordingly, the storage unit (120) and the database (200) may be used interchangeably without being separately distinguished below.
[0190] At least one data collected (or received) from various sources (e.g., a web corpus, a document corpus, a database (DB) website, an API, a server linked to the answer generation system (100), a central server, an external server (300), cloud storage, a user terminal (10), a large dataset, etc.) may be stored in the storage unit (120).
[0191] For example, a plurality of query data collected from the various sources may be stored in the storage unit (120). Here, the query data may be used as training queries to generate a training data set for training a large language model. For example, the query data may include a natural language question that a user can input or an input in the form of a command requesting the performance of a specific task. In this case, the correct answer data for the training query may not be used in the present invention, but the correct answer data for the training query may be stored in the storage unit (120).
[0192] Additionally, a learning data set for training a large language model may be stored in the storage unit (120). Specifically, the storage unit (120) may store a learning data set comprising at least one of a learning query, thought structure data corresponding to the learning query, evaluation data corresponding to the thought structure data, and an evaluation result calculated based on the evaluation data.
[0193] Specifically, the storage unit (120) may store search content retrieved from various sources (e.g., a web corpus, a document corpus, a database (DB) website, an API, a server linked to the answer generation system (100), a central server, an external server (300), cloud storage, a user terminal (10), a large-scale data set, etc.) according to search query data corresponding to a learning query. For example, the search content may include text data in the form of documents, image data, video data, audio data, or structured data (e.g., tables, metadata, etc.), and may be obtained (or collected) through web pages, databases, knowledge graphs, or external APIs. However, the form and method of collection of the search content are not limited to the above examples and may be implemented to include various types of information content.
[0194] Furthermore, a computer program including computer program instructions may be stored in the storage unit (120). Furthermore, a computer program including computer program instructions that control the operation of the answer generation system (100) or control the operation of the control unit (130) when loaded into the processor of the answer generation system (100) may be stored in the storage unit (120).
[0195] Specifically, the storage unit (120) may store data and commands necessary for the operation of the answer generation system (100) according to the present invention. For example, the storage unit (120) may store commands for the operation of a pre-trained large language model (131). The large language model (131) according to the present invention may include at least one artificial intelligence model trained to generate an answer to the query based on at least some of the thought structure data corresponding to the query and the evaluation results of the thought structure data. For example, the large language model (131) may include at least one artificial intelligence model trained to generate an answer to the query based on at least some of the thought structure data corresponding to the query and the evaluation results of the thought structure data, based on at least one large language model among EXAONE, T5 (Text-to-Text Transfer Transformer), BART (Bidirectional and Auto-Regressive Transformer), GPT (Generative Pre-trained Transformer), Gemini, DeepSeek, and LLaMA (Language Model for Many Applications). At this time, the large language model (131) according to the present invention is not limited to the model described above and may further include large language models having the same function.
[0196] As another example, the storage unit (120) may store instructions for the operation of a reference large language model (not shown). Here, the reference large language model is a large language model distinct from the large language model (131) being trained, and may be a large language model used to generate a training data set. As an example, the reference large language model may include at least one of large language models based on EXAONE, T5 (Text-to-Text Transfer Transformer), BART (Bidirectional and Auto-Regressive Transformer), GPT (Generative Pre-trained Transformer), Gemini, DeepSeek, and LLaMA (Language Model for Many Applications).
[0197] At this time, the reference large language model may be included in at least one of the answer generation system (100), the control unit (130), the learning unit (140), and the LLM server (150). In the present invention, the answer generation system (100) or the learning system may be implemented to generate a learning data set using the reference large language model provided from an external server via a network. That is, the storage location or operation method of the reference large language model according to the present invention is not limited to specific examples.
[0198] Meanwhile, the storage unit (120) may store data and commands necessary for the operation of the search module (132). Specifically, the search module (132) is a module for performing Retrieval-Augmented Generation (RAG) and can perform the function of searching for search content corresponding to a learning query (or at least one query) from an external source and utilizing it for answer generation. For example, the search module (132) can generate search query data based on the learning query (or at least one query), search for and acquire (or collect) search content from a database, an external server, a web server, or a vector database, and provide the acquired information to a large language model so that it can be utilized for answer generation. However, the configuration and operation method of the search module (132) are not limited to the above examples and may be implemented to include various forms of content search and acquisition (or collection) methods.
[0199] Additionally, data and commands necessary for the operation of the learning unit (140) may be stored in the storage unit (120). Specifically, the learning unit (140) may refer to a module capable of generating a learning data set for training a large language model to generate an answer to said query based on at least some of the thought structure data corresponding to the query and the evaluation results of the thought structure data, and training the large language model based on the generated learning data set.
[0200] Furthermore, the storage unit (120) may store data and commands necessary for the operation of the prompt generation unit. Specifically, the prompt generation unit may include at least one artificial intelligence model capable of generating a prompt that includes instructions requesting a large language model to perform at least one function. For example, the prompt generation unit may include at least one of EXAONE, T5 (Text-to-Text Transfer Transformer), BART (Bidirectional and Auto-Regressive Transformer), GPT (Generative Pre-trained Transformer), Gemini, DeepSeek, LLaMA (Language Model for Many Applications) based large language models, a rule-based template matching algorithm, a conditional prompting technique, a contextual embedding selection module, and a few-shot prompt generator. At this time, the artificial intelligence model included in the prompt generation unit according to the present invention is not limited to the described models and may further include artificial intelligence models that perform the same function.
[0201] Next, the control unit (130) can perform the role of controlling the overall operation of the answer generation system (100) and the learning system (not shown) related to the present invention. Specifically, the control unit (130) may include a large language model (131). At this time, although the present invention describes the control unit (130) as including a large language model (131), it is not limited thereto. If the large language model (131) exists on an external server (e.g., LLM server (150)), the control unit (130) may interact with the external server to call or control the functions of the pre-trained large language model (131).
[0202] For example, if at least one of the large language model (131) according to the present invention and the reference large language model used for learning the large language model exists in the LLM server (150), the control unit (130) can call or control the functions of the large language model (131) and the reference large language model in conjunction with the LLM server (150). At this time, although the LLM server (150) is described as existing separately from the answer generation system (100) in the present invention, it is not limited thereto, and the answer generation system (100) may be configured to include the LLM server (150).
[0203] That is, the answer generation system (100) and the LLM server (150) according to the present invention may exist separately, or the LLM server (150) may be included in the answer generation system (100). For convenience of explanation, the LLM server (150) and the large language model (131, or reference large language model) may be used interchangeably below, and the use of the large language model (131, or reference large language model) by the control unit (130) can be understood as using at least one large language model included in the LLM server (150).
[0204] For example, the LLM server (150) of the present invention may include at least one large language model based on at least one of EXAONE, T5 (Text-to-Text Transfer Transformer), BART (Bidirectional and Auto-Regressive Transformer), GPT (Generative Pre-trained Transformer), Gemini, DeepSeek, and LLaMA (Language Model for Many Applications). At this time, the large language model included in the LLM server (150) according to the present invention is not limited to the models described above, and the LLM server (150) according to the present invention may include an artificial intelligence model that performs the same function as the large language model.
[0205] The control unit (130) can process signals, data, information, etc. that are input or output through the components of the answer generation system (100) and the learning system described above, or perform a series of data processing to provide or process appropriate information and functions to the user. The control unit (130) can be physically implemented by the processor described above.
[0206] Meanwhile, the control unit (130) can use the learning unit (140) to train the large language model (131) to be trained so that the large language model (131) according to the present invention generates an answer to a query based on at least some of the thought structure data corresponding to the query and the evaluation results of the thought structure data. Specifically, the control unit (130) can use the learning unit (140) to generate a learning data set used for training the large language model (131). At this time, the operation performed by the learning unit (140) according to the present invention can also be described as being performed by the control unit (130).
[0207] Specifically, the learning unit (140) may specify at least one learning query to be used for generating a learning data set. Furthermore, the learning unit (140) may generate thought structure data corresponding to the learning query using a reference large language model. As previously described, the thought structure data according to the present invention may include at least one of a learning query, thought process data corresponding to the learning query, and answer data for the learning query. For example, in the process of generating an answer to a query, the reference large language model may generate thought process data including at least one of inference data, search query data, and search content corresponding to each of a plurality of thought stages.
[0208] The learning unit (140) can extract (or generate) thinking basis data corresponding to thinking process data using a reference large language model. Here, the thinking basis data may refer to basis information that can be referenced or utilized to generate thinking process data during the process of generating answers to learning queries. Furthermore, the learning unit (140) can obtain evaluation data for at least one of the thinking process data and the answer data for the learning query included in the thinking structure data. Specifically, the learning unit (140) can process at least one prompt, which includes at least one instruction requesting the generation of evaluation data for at least one of the thinking process data and the answer data for the learning query, as input to the reference large language model. At this time, the reference large language model can generate evaluation data for at least one of the thinking process data and the answer data for the learning query based on at least one prompt.
[0209] Furthermore, the learning unit (140) may obtain at least one evaluation data from a reference large language model and, based on the obtained evaluation data, perform an evaluation of the thought structure data corresponding to the learning query. Specifically, the learning unit (140) may use the reference large language model to generate an evaluation result for the thought structure data based on the at least one evaluation data. At this time, the evaluation result for the thought structure data may include a final evaluation indicator according to an evaluation indicator corresponding to at least one of the thought process data and the answer data for the learning query included in the thought structure data. Here, the final evaluation indicator may refer to an indicator calculated by comprehensively reflecting the evaluation indicators for the thought process data and the answer data.
[0210] Specifically, the final evaluation indicator is an indicator calculated by integrating the evaluation results regarding the consistency of the underlying relationships and inferences of the thought process data, as well as the accuracy of the response data, and can represent the overall quality of the thought structure data. Furthermore, the final evaluation indicator may be calculated based on a weighted combination, average, or predefined function of multiple evaluation indicators, and may refer to an indicator for quantitatively representing the quality of the thought structure data. However, the calculation method and composition of the final evaluation indicator are not limited to the above examples and may be calculated according to various evaluation criteria and combination methods.
[0211] Furthermore, the learning unit (140) may generate a learning data set including at least one of a learning query, thought structure data corresponding to the learning query, and an evaluation result for the thought structure data. At this time, the learning data set may further include at least one of thought basis data corresponding to the thought structure data, thought process data included in the thought structure data, and evaluation data corresponding to each of the answer data.
[0212] The learning unit (140) can train a large language model (131) to be trained using a learning data set. The large language model training method according to the present invention may be diverse. In this specification, any learning method capable of training a large language model is not limited to any specific type or method. For example, the learning unit (140) can train (or fine-tune) the large language model (131) by updating the parameters of the large language model (131) based on the learning data set so that the large language model (131) generates an answer to the query based on at least some of the thought structure data corresponding to the query and the evaluation results of the thought structure data. Here, the parameters of the large language model (131) include at least one of weight values, bias values, embedding parameters, vision encoder parameters, text encoder parameters, patch embedding parameters, a query / key / value (Q / K / V) matrix of a multi-head attention module, linear transformation coefficients of a feedforward network (FFN), layer normalization coefficients, weights of an attention combination module, an output matrix for calculating token generation logit, scale coefficients and offset coefficients after the activation function, a weight matrix of a multimodal combination module combining image features and text features, a position embedding, a segment embedding, a representation vector of a class token and an image patch token, a learning rate of the vision language model, a dropout rate, and a regularization coefficient. It can mean all learnable parameters.At this time, the parameters of the large language model (131) being updated are not limited to the examples described above, and may include various additional parameters for training (or optimizing, fine-tuning) the large language model.
[0213] At this time, the learning method according to the present invention is not limited to the examples described above and may vary depending on the structure of the large language model (131), learning conditions, objective function setting, or the configuration of the learning data. As an example, the large language model (131) may be learned (or fine-tuned) in at least one of supervised fine-tuning, preference-based tuning, RLHF (Reinforcement Learning from Human Feedback) based fine-tuning, LoRA (Low-Rank Adaptation) based fine-tuning, adapter-based fine-tuning, prompt-tuning, prefix-tuning, and multimodal alignment fine-tuning.
[0214] Meanwhile, the control unit (130) according to the present invention can generate an answer to at least one question by using at least one large language model that has been pre-trained to generate an answer to the question based on at least some of the thought structure data corresponding to the question and the evaluation results of the thought structure data by the learning unit (140).
[0215] Specifically, the control unit (130) may receive at least one query from the user terminal (10). Here, the query (or user query) may refer to request information entered by the user to obtain specific information or solve a problem, and the query may include multimodal input data composed of text data, image data, voice data, video data, or a combination thereof.
[0216] Furthermore, the control unit (130) receives the query from at least one large language model that has been learned in advance.
[0217] Evaluation data corresponding to thought process data and answer data, respectively, can be generated in at least one large language model operated by at least one processor. Based on the evaluation data generated by at least one large language model, an evaluation result for thought structure data corresponding to at least one query can be generated. Furthermore, based on the evaluation result, at least one large language model can generate an answer to at least one query.
[0218] The control unit (130) can obtain an answer to the query from at least one large language model. Furthermore, the control unit (130) can provide the obtained answer to the query to a user terminal. Specifically, the control unit (130) can provide an answer to the query to the user terminal through at least one service page (1000). At this time, the service page (1000) is a user interface displayed on the output unit (or display unit) of the user terminal (10), and may refer to one or more screen configurations for providing an answer to the query and information related to the generation of the answer. For example, the control unit (130) can provide an answer to the query received through the service page (1000) to the user terminal (10) by displaying it in a form including text, a table, a chart, an image, video, audio, or a combination thereof.
[0219] Furthermore, the control unit (130) may provide the evaluation result for the accident structure data on at least one service page along with the answer to the query. For example, the control unit (130) may provide the final evaluation indicator for the accident structure data corresponding to the query on at least one service page along with the answer to the query.
[0220] Additionally, the control unit (130) may provide a modified answer to the query to the user terminal based on the evaluation result. If the evaluation result for the thought structure data satisfies a preset condition, the control unit (130) may generate a modified answer to the query based on the evaluation result using a pre-trained large language model (131). Furthermore, the control unit (130) may obtain the modified answer from the large language model and provide the obtained modified answer to at least one service page.
[0221] Meanwhile, the answer generation system (100) and the learning system (not shown) may include at least one processor, and such processor may include one or more general-purpose processors or one or more special-purpose processors (e.g., digital signal processor, tensor processing unit (TPU), graphics processing unit (GPU), neural network processing unit (NPU), application integrated circuit, application semiconductor (ASIC), etc.). At least one processor may be configured to execute instructions, computer-readable instructions, or other instructions described herein that are stored (or included) in the storage unit (120). Such an answer generation system (100) and the learning system may perform data processing described below in cooperation with memory and at least one processor. The processor may be electrically connected to memory and may perform a series of operations and data processing using data and information stored in memory. Here, “memory” may be a component of the storage unit (120), and “processor” may be used interchangeably with the control unit (130).
[0222] Meanwhile, below, an embodiment of the answer generation system (100) according to the present invention will be described.
[0223] An answer generation system (100) according to one embodiment of the present invention may include at least one memory, at least one processor, at least one pre-trained natural language processing engine, a request processing unit, an inference engine driving unit, and a response output unit. The answer generation system may obtain request information from a user terminal or an external system through an external interface, call a natural language processing engine to produce thought structure data corresponding to the request information and an evaluation result therefrom, determine response information based on at least a portion of the produced evaluation result, and output the result through an external interface.
[0224] The answer generation system (100) may be implemented as a single computing device or as a distributed computing environment in which multiple computing devices are connected by a network. According to one embodiment, the natural language processing engine may be stored directly in the memory of the answer generation system, and according to another embodiment, the natural language processing engine may be stored in an external computing system and implemented in such a way that the answer generation system calls the natural language processing engine through a network.
[0225] That is, the description in this specification that a natural language processing engine is included in an answer generation system can be interpreted to include both cases where the natural language processing engine is physically located inside the answer generation system and cases where the answer generation system is connected to the natural language processing engine in a callable form.
[0226] The memory included in the answer generation system (100) may be configured to store executable commands. The memory may store a pre-trained natural language processing engine and commands that control the invocation of the natural language processing engine. According to one embodiment, the memory may further store at least some of the weight parameters of the natural language processing engine, tokenizer information, input prompt templates, and output parsing rules.
[0227] According to another embodiment, the memory does not directly store weight parameters of the natural language processing engine, but may store at least one of call interface information and call credential information for an external computing system where the natural language processing engine is stored. In this case, the answer generation system can utilize the operation of the natural language processing engine by sending a call request to the external computing system.
[0228] A processor included in the answer generation system (100) may be configured to be connected to memory and to execute instructions stored in memory. The processor may include at least one central processing unit (CPU), at least one graphics processing unit (GPU), at least one neural network processing unit (NPU), or a combination thereof. According to one embodiment, weight operations of the natural language processing engine may be executed on a GPU or an NPU, and instructions of the request processing unit, inference engine driving unit, and response output unit described below may be executed on a CPU or on the same processor as the natural language processing engine.
[0229] The natural language processing engine may be a neural network-based model pre-trained to produce response information corresponding to input information based on at least some of thought structure data corresponding to input information and evaluation results of the thought structure data. The natural language processing engine may also be referred to as a "large language model" in other parts of this specification.
[0230] According to one embodiment, the natural language processing engine may be a decoder-based language model having a Transformer structure, and may be configured to generate a contextual representation of an input token sequence through an attention mechanism and output a probability distribution of the next token from the generated contextual representation. The natural language processing engine may be a model that is pre-trained with a large amount of natural language data and then further fine-tuned to generate response information based on the output and evaluation results of thought structure data.
[0231] According to another embodiment, the natural language processing engine may be composed of at least two distinct model instances. For example, the natural language processing engine may include at least one of a first instance that produces thought structure data from request information and a second instance that produces response information based on the thought structure data and an evaluation result. The first instance and the second instance may share the same pre-training weights or may have different pre-training weights.
[0232] When the natural language processing engine produces thought structure data, the natural language processing engine may output thought process data and response data that can be classified into at least one of inference type, behavior type, and observation type. When the natural language processing engine does not produce thought structure data but produces only response information, the inference engine driver may separate and extract the part corresponding to the thought structure data and the part corresponding to the response information from the output token sequence of the natural language processing engine.
[0233] Additionally, according to one embodiment, the natural language processing engine may also be utilized to produce evaluation results. That is, the natural language processing engine may be called to receive thought structure data and thought basis data as inputs and to output a status label for at least one of inference data, detailed query data, and detailed answer data. According to another embodiment, the evaluation result may be produced by a separate evaluation model distinct from the natural language processing engine.
[0234] The request processing unit may be configured to obtain at least one request information from an external interface. The request processing unit may be implemented as a logical set of instructions among the instructions stored in memory that receive input from an external interface and convert the received input into a format that can be processed by a natural language processing engine.
[0235] According to one embodiment, the request processing unit may perform different input processing procedures depending on the type of external interface. For example, if the external interface is an HTTP-based response protocol, the request processing unit may extract query text included in the request body. If the external interface is based on a messaging queue, the request processing unit may extract query text from messages loaded in the queue. If the external interface is file-based, the request processing unit may read query text from a specified file path in batches or sequentially.
[0236] The request processing unit may perform a preprocessing procedure on the extracted query text. According to one embodiment, the preprocessing procedure may include at least one of encoding normalization, whitespace cleanup, system prompt combining, tokenizer application, and input length limit application.
[0237] When request information is obtained in the form of a bundle of multiple requests rather than a single request, the request processing unit may be configured to separate each of the multiple requests into individual request information and transmit each separated individual request information to the inference engine driver. Additionally, if the request information includes an authentication token or authorization information, the request processing unit may perform a verification procedure for the authentication token first and transmit only the request information that passes the verification procedure to the inference engine driver. Request information that does not pass the verification procedure is not transmitted to the inference engine driver and may be returned as an error response through an external interface.
[0238] The inference engine driver may be configured to transmit request information to a natural language processing engine so that inference operations based on at least some of the thought structure data and evaluation results are performed. The inference engine driver may be implemented as a logical set of instructions regarding input prompt configuration, model calling, output parsing, and thought structure data formatting, among the instructions that control the invocation of the natural language processing engine stored in memory.
[0239] According to one embodiment, the inference engine driver may be configured to call a natural language processing engine once with request information as input, and to obtain thought structure data including both thought process data and answer data from the single call. According to another embodiment, the inference engine driver may be configured to call a natural language processing engine multiple times and to accumulate or produce thought process data and answer data in stages from the results of the multiple calls.
[0240] The inference engine drive unit can classify each thought process data included in the thought structure data obtained from the output of the natural language processing engine into any one of the inference type, behavior type, and observation type. The classification procedure includes markers included in the text format or structural format of the thought process data (e.g., " <think> ", " <search> ", " <information>It can be performed based on markers such as ", and can also be performed by additionally calling a natural language processing engine or a separate classification model.
[0241] If the evaluation result has already been produced prior to the time of the natural language processing engine call, the inference engine driver can combine the evaluation result with the input prompt of the natural language processing engine to form the call input. If the evaluation result has not been produced at the time of the natural language processing engine call, the inference engine driver can obtain only the thought structure data in the first call, wait for the evaluation result to be produced, and combine the produced evaluation result to perform the second call.
[0242] Additionally, the inference engine driver may be configured to retry a call if the call to the natural language processing engine fails or if the output does not conform to a preset output schema. The number of call retries may be limited by a preset maximum number of retries, and if the output does not conform to the output schema even after reaching the maximum number of retries, the inference engine driver may transmit an error signal to the response output unit.
[0243] The response output unit may be configured to output response information corresponding to the request information from the output of the natural language processing engine and to transmit the output response information to a user terminal through an external interface. The response output unit may be implemented as a logical set of commands among the commands stored in memory regarding post-processing of the response information, serialization to an external interface, and transmission of the response.
[0244] According to one embodiment, the response output unit may provide at least one of accident structure data and evaluation results together with response information through an external interface. For example, the response output unit may serialize the response information into a body area and at least one of the accuracy indicator and evidence consistency indicator included in the evaluation result into a metadata area and transmit them to an external interface. According to another embodiment, the response output unit may transmit only the response information through an external interface, and transmit the accident structure data and evaluation results only to a log storage within the response generation system (100) or a separate analysis system.
[0245] The response output unit may perform a post-processing procedure before transmitting response information to an external interface. According to one embodiment, the post-processing procedure may include at least one of token restoration, detoxification, markup cleanup, and length limit application.
[0246] The response output unit may transmit a modified response obtained from a natural language processing engine or another natural language processing engine distinct from it to an external interface, instead of or together with the response information, if the evaluation result satisfies preset conditions. For example, if the accuracy indicator included in the evaluation result is determined to be a value that does not correspond to the correct answer, the response output unit may request the inference engine driver to produce a modified response, and the inference engine driver may re-call the natural language processing engine to produce a modified response by combining the evaluation result with the input prompt of the natural language processing engine. The response output unit may transmit the produced modified response to a user terminal through an external interface.
[0247] In addition, the response output unit can adaptively determine the method of delivering response information based on the availability status of the external interface. If the external interface requires an immediate response, the response output unit can deliver the response information synchronously. If the external interface allows an asynchronous response, the response output unit can deliver the response information asynchronously based on a queue.
[0248] In the above description, an embodiment of the answer generation system (100) according to the present invention has been described. Below, an embodiment of the large language model learning system according to the present invention will be described.
[0249] A large language model learning system according to one embodiment of the present invention may include at least one memory, at least one processor, at least one pre-trained reference natural language processing engine, at least one tuning target natural language processing engine to be trained, a training request processing unit, an inference flow collection unit, a verification processing unit, and a parameter update unit. The large language model learning system may receive training request information corresponding to a training query, obtain thought structure data from the reference natural language processing engine, produce an evaluation result for the thought structure data, and update the parameters of the tuning target natural language processing engine using at least a portion of the produced evaluation result and the thought structure data.
[0250] A large language model training system may be implemented on a computing system that is physically identical to the answer generation system, or on a separate computing system that is physically separated. According to one embodiment, a natural language processing engine subject to tuning that has been updated in the large language model training system may be deployed to and utilized as the natural language processing engine of the answer generation system after the update is completed.
[0251] Memory included in a large language model training system can be configured to store executable instructions. The memory can store a pre-trained reference natural language processing engine and a natural language processing engine to be tuned, and can additionally store instructions that control the invocation of the reference natural language processing engine and instructions that control the parameter updates of the natural language processing engine to be tuned.
[0252] The relationship between the two natural language processing engines stored in memory can be configured in various ways according to one embodiment. According to the first embodiment, the reference natural language processing engine and the natural language processing engine to be tuned may be two instances sharing the same pre-trained weights, and only the weights of the natural language processing engine to be tuned may be updated during the training phase. According to the second embodiment, the reference natural language processing engine and the natural language processing engine to be tuned may be separate models having different model architectures or different pre-trained weights. According to the third embodiment, the reference natural language processing engine may not be stored directly in the memory of the large language model training system but may be stored in an external computing system and called through a call interface, and only the natural language processing engine to be tuned may be stored directly in memory.
[0253] The above call control instructions and parameter update control instructions may be stored separately in different memory regions within memory, or may be stored mixed within the same memory region. According to one embodiment, the call control instructions may be composed of instructions corresponding to an inference path, and the parameter update control instructions may be composed of instructions corresponding to a training path. The instructions of the inference path may be configured to perform forward operations that do not change the weights of the natural language processing engine, and the instructions of the training path may be configured to perform backward operations based on a loss function and weight update operations based on an optimizer.
[0254] The training request processing unit may be configured to acquire at least one training request information and transmit the acquired training request information to a reference natural language processing engine so that inference operations are performed. The training request information is a training query used in the training phase and may be acquired from at least one of an external data repository, a user input interface, or a pre-configured training data set.
[0255] According to one embodiment, the training request processing unit may acquire a plurality of training request information included in a training data set in a batch or in batch units and load them into a queue so that they can be processed in batches by the inference flow collection unit and the verification processing unit. According to another embodiment, the training request processing unit may be configured to acquire training request information dynamically generated from a policy model in real time during the reinforcement learning stage and to perform inference operations immediately on the acquired training request information.
[0256] The training request processing unit can perform a preprocessing procedure suitable for the learning stage on the training request information. According to one embodiment, the preprocessing procedure may include at least one of combining system prompts, combining instructions in the thought flow output format, applying a tokenizer, and applying an input length limit.
[0257] The inference flow collection unit may be configured to collect thought structure data corresponding to training request information from the output of a reference natural language processing engine. The thought structure data may include at least one of intermediate inference expression and response expression information. According to one embodiment, the inference flow collection unit may separate a part corresponding to thought process data and a part corresponding to answer data from the output token sequence of the reference natural language processing engine. The separation is performed using a preset marker (e.g., " <think> ...< / think> ", " <search> ...< / search> ", " <information> ...< / information> ", " <answer> ...< / answer> It can be performed based on the " (markers) etc.
[0258] The inference flow collection unit can formalize the internal structure of the thought structure data by classifying each separated thought process data into one of the inference type, behavior type, and observation type. In the formalization process, the inference flow collection unit may call up and apply an alias mapping table from memory to map different markers to the same type. The alias mapping table may also be referred to as a "type classification table," "alias map," or "step type canonicalization map" in other parts of this specification.
[0259] The structured thought structure data collected by the inference flow collection unit can be transmitted to the verification processing unit and can also be stored in memory or an external data storage as part of the training data set.
[0260] The verification processing unit may be configured to acquire verification output data for at least one of accident process data and response data, and to execute a verification procedure for accident structure data based on the acquired verification output data to produce verification result information. The verification output data may also be referred to as "evaluation data" in other parts of this specification, and the verification result information may also be referred to as "evaluation result" in other parts of this specification.
[0261] According to one embodiment, the verification output data produced by the verification processing unit may include at least one of the following intermediate outputs. The verification processing unit may call a reference natural language processing engine to produce accident basis data related to at least some of the training request information and response data from the accident process data.
[0262] The verification processing unit can calculate a first evaluation indicator corresponding to each inference data included in the accident process data based on the accident basis data, and combine the calculated first evaluation indicators to produce first evaluation data for the accident process data.
[0263] The verification processing unit can generate at least one detailed query data by decomposing it from training request information and at least one detailed answer data by decomposing it from answer data, and can generate second evaluation data by performing a judgment on whether the answer data includes an answer for each detailed query data and whether it is correct based on the accident basis data.
[0264] The verification processing unit specifies a fourth state label for each detailed answer data based on accident basis data, calculates at least one of a second evaluation indicator and a third evaluation indicator for the answer data based on the specified fourth state label, and can calculate a third evaluation data for the answer data based on the calculated evaluation indicators.
[0265] The verification processing unit may combine at least some of the first evaluation data, second evaluation data, and third evaluation data calculated above to produce verification result information for accident rescue data. The verification result information may include a final evaluation indicator for the accident rescue data, and the final evaluation indicator may also be referred to as a "reward score" or "reward indicator" in other parts of this specification.
[0266] The verification result information produced by the verification processing unit can be utilized as a learning signal by the parameter update unit described later, and can also be utilized as information provided together to the user terminal by the response output unit of the answer generation system.
[0267] According to one embodiment, the verification processing unit may generate verification output data by additionally calling a separate evaluation model that can be utilized as a natural language processing engine. According to another embodiment, the verification processing unit may generate verification output data by reusing a reference natural language processing engine as an evaluation model.
[0268] Additionally, the verification processing unit may determine whether to execute an additional search procedure based on the sufficiency of the incident basis data. If there exists detailed answer data that has not been sufficiently verified by the incident basis data or detailed query data that has not been sufficiently resolved, the verification processing unit may convert the text representation of the said detailed answer data or detailed query data into a search query and call an external search function to obtain additional information. Additional incident basis data may be generated from the obtained additional information, and the generated additional incident basis data may be added to the set of incident basis data. This additional search procedure may be performed until a preset maximum number of iterations is reached, and the additional search procedure may not be performed if at least one of the detailed answer data is determined to be in a contradictory state with the incident basis data.
[0269] The parameter update unit may be configured to update the parameters of a natural language processing engine to be tuned using at least some of the verification result information and thought structure data. The parameter update unit may be implemented as a logical set of instructions regarding the configuration of a training data set, evaluation of a loss function, inverse operation, and application of an optimizer among the parameter update control instructions stored in memory.
[0270]
[0271] According to one embodiment, the parameter update unit may construct a training data set using the final evaluation indicator, training request information, and thought structure data included in the verification result information. The training data set may include mapping information between the training request information and the thought structure data, and may be composed of a subset of thought structure data selected from among the thought structure data suitable for training.
[0272] According to another embodiment, the parameter update unit can update the parameters of the natural language processing engine to be tuned using a reinforcement learning method that uses the final evaluation indicator included in the verification result information as a reward signal. As an example of the reinforcement learning method, the parameter update unit can determine a preferred thought structure and a non-preferred thought structure based on the accuracy indicator and the reward indicator included in the verification result information among two thought structure data, and can adjust the parameters of the natural language processing engine to be tuned based on the pairs of the preferred thought structure and the non-preferred thought structure.
[0273] The parameter update unit can select accident structure data as a training target in the step of configuring a training data set, wherein the accuracy indicator included in the verification result information is a value corresponding to the correct answer and the evidence consistency indicator is greater than or equal to a preset threshold. According to one embodiment, accident structure data in which the evidence consistency indicator is less than a preset threshold even if the accuracy indicator corresponds to the correct answer may be excluded from the training data set, and accident structure data in which the accuracy indicator does not correspond to the correct answer may also be excluded from the training data set.
[0274] In addition, the parameter update unit can dynamically adjust the update procedure according to the progress of the learning phase. For example, in the early learning phase, only the accuracy metric among the validation result information can be used as a learning signal; in the mid-learning phase, both the accuracy metric and the reward metric can be used as a learning signal; and in the late learning phase, the reward metric can be used as a learning signal with a greater weight assigned to it.
[0275] Additionally or alternatively, the parameter update unit may deploy the updated tuning target natural language processing engine as a new natural language processing engine to replace the answer generation system's natural language processing engine. The deployment procedure may include an operation to replace the natural language processing engine stored in memory with the updated tuning target natural language processing engine, and may also include an operation to update call interface information so that the answer generation system calls the updated tuning target natural language processing engine.
[0276] In the above description, the answer generation system (100) and the learning system of the present invention have been described, and this can implement the answer generation method and the large language model learning method described below.
[0277] Hereinafter, a large language model learning method according to the present invention will be described in more detail with reference to FIGS. 6a to 6d, FIGS. 7a, FIG. 7b, FIGS. 8a to 8d, FIGS. 9a to 9e, and FIGS. 10a to 10d. FIGS. 6a to 6d are conceptual diagrams for explaining the process of generating thought structure data corresponding to a query according to the present invention, FIGS. 7a, FIGS. 7b, FIGS. 8a to 8d, and FIGS. 9a to 9e are conceptual diagrams for explaining the process of evaluating thought structure data according to the present invention, and FIGS. 10a to 10d are conceptual diagrams and mathematical formulas for explaining the process of generating thought structure data and performing an evaluation of thought structure data according to the present invention.
[0278] Furthermore, the process of generating thought structure data and the process of evaluating thought structure data illustrated in FIGS. 6a to 6d, FIGS. 7a, FIGS. 7b, FIGS. 8a to 8d, FIGS. 9a to 9e, and FIGS. 10a to 10d may also be applied to the answer generation method (inference step) according to the present invention. However, in the answer generation method, a pre-trained large language model (131) may perform the generation and evaluation of thought structure data instead of the standard large language model (160) of the learning step, and this difference can be understood by referring to FIGS. 11a to 11d and the description regarding the answer generation method.
[0279] Furthermore, according to the present invention, at least one of the thought process evaluation result and the answer candidate evaluation result can be obtained based on thought basis data extracted from thought process data by a language model. The thought basis data may include at least one reference data serving as a criterion for obtaining at least one of the thought process evaluation result and the answer candidate evaluation result, and the reference data may be obtained from at least one external data source. The external data source may include at least one of a search engine and a knowledge database, and may also include at least one of a document corpus, a web page, API response data, and an external knowledge base.
[0280] As previously explained, the answer generation system (100) according to the present invention may be configured to include a learning system. In this case, the learning system may provide various services related to the present invention (e.g., generation of a learning data set and learning of a large language model) through the communication unit (110), storage unit (120), control unit (130), and learning unit (140) of the answer generation system (100). As an example, the learning unit (140) of the answer generation system (100) may refer to the learning system.
[0281] In the present invention, the answer generation system (100) is described as including a learning system (or a learning unit (140)) that performs learning on a large language model (131) to be learned to generate an answer to said question based on at least one thought structure data corresponding to said question and at least some of the evaluation results of said thought structure data, but is not limited thereto. The learning system (or a learning unit (140)) may be linked with an external server or a cloud computing environment to generate a learning data set for learning the large language model and to perform learning of the large language model. At this time, the large language model (131) learned by the large language model learning method according to the present invention may be used in the answer generation system (100).
[0282] That is, the answer generation system (100) according to the present invention may include a learning system, or the learning system may exist as a separate and independent system from the answer generation system (100). For convenience of explanation, the answer generation system (100) is described below as including a learning system, and the learning system performs a large language model learning method through at least one of the communication unit (110), storage unit (120), control unit (130), and learning unit (140) of the answer generation system (100). Additionally, the answer generation system (100) and the learning system may be used in combination, and in this case, the answer generation system (100) may be described as performing the function of the learning system.
[0283] According to the present invention, an integrated configuration in which the answer generation system (100) includes a learning system, and a separated configuration in which the learning system is separated from the answer generation system (100) into a separate computing system, may both be included within the scope of the rights of the present invention. According to one embodiment, in the integrated configuration, at least one of the communication unit (110), storage unit (120), control unit (130), and learning unit (140) of the answer generation system (100) may be implemented to perform the functions of the learning system together, and in the separated configuration, the learning system may be implemented to be communicationally connected to the answer generation system (100) via a network to independently perform the generation of a learning data set and the learning of a large language model, and then provide the learned large language model (131) to the answer generation system (100). Furthermore, in the separated configuration, the learning system may be implemented to operate on at least one of an external server, a cloud computing environment, a distributed computing environment, or a separate training computing system (1050).
[0284] Furthermore, the operation performed by the learning unit (140) in the present invention can also be understood as an operation performed by the control unit (130), and for convenience of explanation, each step of the large language model learning method can be understood as being performed by at least one of the learning unit (140) and the control unit (130). Below, the process of generating (or constructing) a learning data set used for learning a large language model according to the present invention will be described.
[0285] In the present invention, a step of processing at least one learning query as input to at least one pre-trained reference large language model operated by at least one processor may be performed.
[0286] Specifically, the learning unit (140) may specify at least one learning query among multiple query data already stored in the storage unit (120) and generate a learning data set using the specified learning query. Specifically, the learning unit (140) may specify at least one learning query to be included in the learning data set by selecting at least one of multiple query data collected from various sources (e.g., a web corpus, a document corpus, a database (DB) website, an API, a server linked to the answer generation system (100), a central server, an external server (300), cloud storage, a user terminal (10), a large-scale data set, etc.). At this time, correct answer data may be matched to each of the multiple query data.
[0287] As illustrated in FIG. 6a, the learning unit (140) can process at least one learning query (600) as input to a reference large language model (160). Here, the reference large language model (160) may refer to a large language model that operates by at least one processor and is capable of generating data necessary to generate a learning data set corresponding to at least one learning query (600). At this time, the reference large language model (160) according to the present invention may be a large language model used as a teacher model and may refer to at least one of a pre-trained general-purpose large language model and a fine-tuned large language model for a specific domain.
[0288] Additionally, the reference large language model (160) may be a single large language model, but it may also be configured to generate various data for creating a training data set by combining multiple large language models. However, the type, configuration, and learning method of the reference large language model (160) are not limited to the above examples and may be implemented to include various forms of large language models.
[0289] Next, in the present invention, a step of obtaining thought structure data corresponding to at least one learning query from a reference large language model may be performed.
[0290] A reference large language model (160) according to the present invention can generate at least one thought structure data (610) corresponding to a learning query in conjunction with a search module (132). Here, as illustrated in (a) of FIG. 10a, the thought structure data may include at least one of a learning query (600), a thought process data (620) corresponding to the learning query (600), and an answer data (630).
[0291] Specifically, the reference large language model (160) can generate thought process data (620) for generating an answer corresponding to the above-mentioned learning query based on the search content. Here, the thought process data (620) may be composed of a multi-stage structure including a thinking stage, an information search stage, and an information utilization stage (observation or information) performed to generate an answer to the query, and may include data related to the generation of an answer generated at each stage.
[0292] As illustrated in FIG. 6b, a reference large language model (160) may receive at least one learning query (600) as input and generate inference data (621a, 621b) for generating an answer to the query. Here, the inference data (621a, 621b) may refer to data representing a thought flow derived through a step-by-step thought process to generate an answer to the learning query. For example, to generate an answer to the learning query ("Who was born first, William March or Richard Brautigan?"), the inference data may generate at least one inference data ("To determine who was born first, I need to compare the birth years of William March and Richard Brautigan." (621a), "From the search results, I found that William March was born in 1923, and Richard Brautigan was born in 1935. Therefore, William March was born first." (621b).
[0293] Furthermore, the reference large language model (160) can generate at least one search query data (622) associated with at least one inference data. For example, the reference large language model (160) can generate at least one search query ("William March birth year, Richard Brautigan birth year" (622) in relation to at least one inference data ("To determine who was born first, I need to compare the birth years of William March and Richard Brautigan." (621a)). However, the order, method, and configuration of generating the inference data and search query data are not limited to the above examples and can be implemented to include various forms of inference and information search processes.
[0294] Furthermore, the standard large language model can obtain search content (623) for generating an answer corresponding to the learning query from various sources, including at least one of a database (200) and an external server (300), based on search query data (622) through a search module (132). Here, the search content (623) may include text data in the form of documents, image data, video data, audio data, and structured data (e.g., chart, graph, table, metadata, etc.). However, the form and method of collection of the search content are not limited to the above examples and may be implemented to include various types of information content.
[0295] For example, the search module (132) can search for documents, text content, images, videos, or structured data related to the search query data (622) from a database (200), a web server, a document storage server, a search engine server, a vector database server, or an external knowledge provision API server, based on the search query data, and obtain (or collect) them as search content.
[0296] Additionally, the search module (132) may select (or sort) at least some of the searched multiple search contents based on their relevance (or similarity) to the search query data and provide them to the reference large language model (160). For example, the search module (132) may select the top K search results (top-K) based on the similarity between the search query data and the multiple search contents, wherein the similarity may be calculated based on at least one of keyword-based similarity and embedding-based vector similarity. For example, the similarity may be calculated based on cosine similarity, Manhattan distance, Euclidean distance, or inner product, etc.
[0297] Additionally, the search module (132) can obtain (or collect) at least one search content corresponding to a learning query by performing re-ranking on a plurality of selected search results. However, the method of selecting and sorting search results is not limited to the above example and can be implemented based on various criteria and algorithms. The method of operation of the search module (132) and the method of obtaining search content according to the present invention are not limited to the above example and can be implemented to include various forms of information search and selection methods.
[0298] Furthermore, the reference large language model (160) can generate thought process data (620) including at least one inference data (621a, 621b), at least one search query data (622), and at least one search content (623). Furthermore, the reference large language model (160) can generate answer data (630) for the learning query based on the thought process data. For example, the reference large language model (160) can generate answer data (630) for the learning query (600) based on at least one inference data (621a, 621b), at least one search query data (622), and at least one search content (623) included in the thought process data.
[0299] Through this, it can be understood that a standard large language model (160) operated by at least one processor can generate thought structure data (610) including at least one of a learning query (600), thought process data (620) corresponding to the learning query, and answer data (630) for the learning query. However, the method of generating and configuring the thought structure data according to the present invention is not limited to the above examples and can be implemented to include various forms of inference structures and information utilization methods.
[0300] In the above, a process was described for processing at least one training query as input to at least one pre-trained reference large language model operated by at least one processor, and for obtaining thought structure data corresponding to the at least one training query from the reference large language model.
[0301] Below, we will explain the process of generating at least one thought basis data used to verify thought structure data corresponding to a learning query.
[0302] As illustrated in FIG. 6c, the learning unit (140) can generate (or extract) at least one thought basis data (660) related to at least some of the learning query (600) and the answer data (630) for the learning query (600) from the thought process data (620) using a reference large language model (160). Here, the thought basis data (660) may refer to basis information that can be referenced or utilized to generate the thought process data (620) corresponding to the learning query (600). As illustrated in FIG. 10a (a), the thought basis data (660) may be configured to include a first thought basis data, a second thought basis data, etc.
[0303] The accident basis data disclosed in this specification can function as intermediate state data that is used as an evaluation criterion in the step of calculating the evaluation result for at least one of the accident process data and the answer candidate data, as well as being utilized to determine the update path of the answer candidate data. According to one embodiment, the accident basis data may be obtained from at least one external data source, may be directly extracted from the accident process data, or may be generated by combining the accident process data with information from the external data source. Such accident basis data can provide a technical effect that can improve the explainability and verifiability of the answer generation process by enabling the external presentation of the basis for answer generation.
[0304] Specifically, the thought basis data (660) may include information such as documents, sentences, and atomic fact information related to the learning query. Here, atomic fact information may refer to a minimum unit of factual representation (e.g., structured information in the form of a triple) that includes specific objects, attributes, and relationships. Additionally, the thought basis data (660) may consist of not only text data but also multimodal data such as structured data, tables, nodes of knowledge graphs, relationship information, images, and videos, and may be selectively referenced at each stage of the thought process. However, the composition, scope, and generation method of the thought basis data (660) are not limited to the above examples and may be implemented as various forms of information sets.
[0305] For example, the learning unit (140) may process as input to a reference large language model a prompt (or a first prompt) that includes at least one instruction requesting to extract (or generate) at least one thought basis data from the thought process data, along with thought process data and a learning query. As illustrated in FIG. 6d (a), the first prompt (640) may include at least one system prompt (or a first system prompt, 641) that includes at least one instruction requesting to extract at least one thought basis data (e.g., detailed factual information) related to generating answer data for a learning query based on the learning query, search query data, and search content.
[0306] Specifically, the first system prompt (641) may include at least one instruction requesting that detailed factual information related to a learning query be extracted from search content, such that the detailed factual information is expressed in the form of a single sentence and includes only information that is clearly supported by thought process data. Additionally, the first system prompt (641) may include at least one instruction requesting that attributes such as time information (e.g., year, date) be included if they are explicitly mentioned in the thought process data, or that relational information such as causal or mechanical relationships be included only if they are clearly expressed.
[0307] Furthermore, the first system prompt (641) may include at least one instruction requesting that only at least one thought-based data related to the learning query be selectively extracted, that search content related to the learning query be given priority consideration, and that the search query data be restricted to being used as auxiliary contextual information. Additionally, the first system prompt (641) may include content requesting that at least one thought-based data be provided in the form of a citation from the original text, or that inference regarding relationships not explicitly expressed be excluded. However, the configuration and content of the first system prompt (641) included in the first prompt (640) are not limited to the above examples and may be implemented to include various forms of information extraction and constraints.
[0308] Additionally, as illustrated in FIG. 6d (b), the first prompt (640) may include a user prompt (or, a first user prompt, 642) that extracts reasoning data (660) based on information obtained from learning queries and search content.
[0309] For example, the first user prompt (642) may be configured to include a learning query, search query data (622) related to the learning query, and search content (623). Additionally, the first user prompt (642) may additionally include at least one past accident basis data (e.g., previously extracted detailed fact information, 650)) extracted from the accident process data. Here, the at least one past accident basis data (650) may refer to accident basis data generated in a processing unit performed prior to the current processing unit among a plurality of processing units in the accident process data (620). At this time, the first user prompt (642) is not necessarily required to include the past accident basis data (650), but may be configured to additionally include it only if there is past accident basis data (650) corresponding to the learning query (600).
[0310] The configuration of the first user prompt (642) according to the present invention and the types of information included are not limited to the above examples and can be implemented to include various forms of query information, search information and previous processing results.
[0311] Furthermore, according to the present invention, at least one of the thought process evaluation result and the answer candidate evaluation result can be obtained based on thought basis data extracted from thought process data by a language model. The thought basis data may include at least one reference data serving as a criterion for obtaining at least one of the thought process evaluation result and the answer candidate evaluation result, and the reference data may be obtained from at least one external data source. The external data source may include at least one of a search engine and a knowledge database, and may also include at least one of a document corpus, a web page, API response data, and an external knowledge base.
[0312] Additionally, according to the present invention, at least one of the thought process evaluation result and the answer candidate evaluation result may be obtained based on at least one state label determined by a language model based on thought basis data. The state label may include a plurality of different labels indicating whether the thought basis data matches at least one of the thought process data and the answer candidate data. According to one embodiment, the plurality of different labels may include a first label indicating that the thought basis data matches at least one of the thought process data and the answer candidate data, and a second label indicating that the thought basis data does not match at least one of the thought process data and the answer candidate data. If the state label is the first label, the answer may include the answer candidate data, and if the state label is the second label, the answer may include a result in which the answer candidate data is updated based on at least one of the thought process evaluation result and the answer candidate evaluation result. Through such label-based branching processing, the answer candidate data may be adopted as the answer in the case of a matching label, and an update based on the evaluation result may be performed in the case of a mismatch label.
[0313] Furthermore, the learning unit (140) can obtain at least one thought basis data (660) corresponding to the thought process data (620) from the standard large language model (160).
[0314] In the above, the process of generating at least one thought basis data used to verify thought structure data corresponding to a learning query was described.
[0315] In the following, we will describe the process of obtaining evaluation data for at least one of the at least one thought process data and the answer data to the learning query included in the thought structure data.
[0316] In the present invention, a step of obtaining evaluation data for at least one of at least one thought process data and answer data for a learning query included in thought structure data may be performed.
[0317] As illustrated in FIG. 7a, the learning unit (140) can generate first evaluation data (710) for thought process data (620) using a reference large language model (160). Specifically, the learning unit (140) can generate first evaluation data (710) corresponding to each of at least one inference data (621a, 621b) included in the thought process data (620) using the reference large language model (160). For example, the learning unit (140) can generate first evaluation data (710) related to whether at least one inference data (621a, 621b) satisfies logical consistency with at least one thought basis data (660) using the reference large language model (160). Here, logical consistency may mean whether the inference data is consistently derived based on the thought basis data, or whether it has a relationship supported or backed by the thought basis data. Specifically, logical consistency may include whether inferential data can be reasonably derived without contradicting the factual information contained in the reasoning basis data, and it may be a concept that encompasses cases where it is sufficiently supported by the reasoning basis data, partially supported, or unsupported or contradictory.
[0318] More specifically, the learning unit (140) can use a reference large language model (160) to calculate a first evaluation index corresponding to each of at least one inference data (621a, 621b) included in the thought process data (620), based on at least a portion of the learning query (600) and the thought basis data (660). Furthermore, the learning unit (140) can use the reference large language model (160) to generate a first evaluation data (710) for the thought process data (620) based on the first evaluation index corresponding to each of the at least one inference data.
[0319] For example, the learning unit (140) may process as input to the reference large language model (160) a prompt (or, second prompt, 700) which includes at least one instruction requesting to calculate a first evaluation indicator corresponding to each of at least one inference data (621a, 621b) included in the thought process data (620) based on at least a portion of the learning query (600) and the thought basis data (660). Here, the first evaluation indicator may be calculated based on a first state label determined to correspond to each of the at least one inference data among a plurality of preset state labels (or a plurality of thought process state labels) based on the thought basis data.
[0320] As illustrated in (a) of FIG. 7b, the second prompt (700) may include a second system prompt (701) that includes at least one instruction requesting to determine whether at least one inference data (621a, 621b) included in the thought process data (620) satisfies logical consistency based on the thought basis data (660).
[0321] Specifically, the second system prompt (701) may include instructions requesting that the logical validity of each of at least one inference data (621a, 621b) be determined based on the accident basis data (660), using the learning query (600), accident basis data (660), and inference data (621a, 621b) as inputs. At this time, the second system prompt (701) may include instructions requesting that only the accident basis data (660) be considered as reliable information and that inference regarding content not clearly supported by the accident basis data (660) be restricted.
[0322] Additionally, the second system prompt (701) may include a directive requesting the selection of one of a plurality of state labels (or a plurality of accident process state labels, Ex: "supported", "partially_supported", "contradicted" and "unknown") depending on whether the inference data is supported by the accident basis data (660), partially supported, contradicts the accident basis data, or is difficult to determine, and the selected state label may be determined as a first state label corresponding to each of the inference data (621a, 621b).
[0323] At this time, the second system prompt (701) may include a judgment criterion for each of the multiple state labels (or multiple accident process state labels), and accordingly, may include at least one instruction requesting the reference large language model (160) to select a first state label corresponding to the inference data (621a, 621b). Additionally, the second system prompt (701) may include at least one instruction requesting identification of any parts of the content included in the inference data (621a, 621b) that are not supported by the accident basis data (660), or not to judge as a separate error when content identical to information already included in the accident basis data (660) is simply restated. However, the configuration of the second system prompt (701) and the content of the included instructions are not limited to the above examples and may be implemented to include various forms of inference evaluation criteria and judgment conditions.
[0324] Additionally, as illustrated in (b) of FIG. 7b, the second prompt (700) may include a user prompt (or, a second user prompt, 702) comprising at least one of a learning query (600), at least one reasoning basis data (660), and inference data to be evaluated. Specifically, the second user prompt (702) may be configured in an input format necessary to determine the relationship between the reasoning basis data (660) for each of the at least one inference data (621a, 621b). However, the configuration and format of the information included in the second user prompt (702) are not limited to the above examples and may be implemented to include various forms of learning queries, reasoning basis data, and inference data.
[0325] Furthermore, in the reference large language model (160), based on the second prompt (700), a first state label corresponding to each of at least one inference data (621a, 621b) included in the thought process data (620) can be specified (or determined, selected). Furthermore, in the reference large language model (160), based on the specified first state label, a first evaluation index corresponding to the at least one inference data can be calculated.
[0326] As illustrated in FIG. 10b (a), the first evaluation indicator may be defined as a label value corresponding to the first state label. Specifically, the first evaluation indicator may be set to have different values depending on the first state label determined corresponding to each of at least one inference data. For example, in the reference large language model (160), if the first state label corresponding to the inference data is the first label (Ex: "supported") among a plurality of state labels (or a plurality of thought process state labels), a first value (Ex: 1.0) may be calculated. Additionally, in the reference large language model (160), if the first state label corresponding to the inference data is the second label (Ex: "partially_supported"), a second label value (Ex: 0.5) may be calculated, and if the first state label is the third label (Ex: "contradicted") or the fourth label (Ex: "unknown"), a third label value (Ex: 0.0) may be calculated.
[0327] That is, the first evaluation indicator can be defined as a value that quantitatively indicates the extent to which inference data is supported by reasoning data. However, the magnitude and range of the value corresponding to the first evaluation indicator are not limited to the examples and can be set to various numerical ranges or scales.
[0328] Furthermore, the standard large language model (160) can calculate a thought verification index corresponding to the thought process data based on a first evaluation index corresponding to each of at least one inference data included in the thought process data (620).
[0329] As illustrated in (b) of FIG. 10b, the reference large language model (160) can calculate the thought verification index by summing the first evaluation index corresponding to each of the plurality of inference data included in the thought process data (620) and dividing by the number of inference data to calculate the average value. That is, the thought verification index can be defined as an average value that comprehensively reflects the evaluation results of at least one inference data included in the thought process data (620), and can be an index (or score) representing the overall reliability of the thought process data.
[0330] In the reference large language model (160), first evaluation data can be generated including a first state label corresponding to each of at least one inference data included in the thought process data (620), a first evaluation indicator, and a thought verification indicator corresponding to the thought process data. Furthermore, the learning unit (140) can obtain first evaluation data corresponding to at least one thought process data (620) included in the thought structure data (610) from the reference large language model (160).
[0331] Furthermore, the learning unit (140) can obtain evaluation data corresponding to the answer data (630) for the learning query (600) included in the thought structure data (610) by using the standard large language model (160). Specifically, as illustrated in FIG. 8a, the learning unit (140) can generate at least one detailed query data (810) from the learning query (600) by using the standard large language model (160). Here, the detailed query data is query data generated by breaking down the learning query (600) into multiple detailed queries, and may refer to a query that expresses the complex information request included in the learning query by separating it into each individual fact or relationship unit. For example, if the learning query (600) is "When and where was the Eiffel Tower built, and what is its purpose?", the detailed query data (810) may include a query broken down by detailed information items constituting the learning query (600), such as "Eiffel Tower construction time (C1)", "Eiffel Tower construction location (C2)", and "Eiffel Tower construction purpose (C3)". In this way, as illustrated in (c) of FIG. 10a, the detailed query data (810) may include a first detailed query data, a second detailed query data, and the like.
[0332] Specifically, the learning unit (140) may process as input to the reference large language model (160) a prompt (or a third prompt, 800) that includes at least one instruction requesting to generate (or extract) at least one detailed query data from the learning query (600). For example, as illustrated in FIG. 8b, the third prompt (800) may include a system prompt (or a third system prompt, 801) that includes at least one instruction requesting to extract explicitly expressed detailed query data from the learning query (600).
[0333] Specifically, the third system prompt (801) may include at least one instruction requesting the selective extraction of only the detailed query data that the answer to the learning query must satisfy, excluding information corresponding to the background explanation included in the learning query (600). Additionally, the third system prompt (801) may include at least one instruction requesting that each detailed query data be configured in a form verifiable by the reasoning data, while restricting the generation of assumptions or formal requirements that are not explicitly expressed. However, the configuration of the third system prompt (801) and the content of the instructions are not limited to the above examples and may be implemented to generate detailed query data in various forms.
[0334] Additionally, the third prompt (800) may include a user prompt (or a third user prompt, 802) that extracts detailed query data based on a learning query and generates it in a structured form. Specifically, the third user prompt (802) may include a learning query (600) and may include instructions that request at least one detailed query data to be extracted (or generated) from the learning query and returned said detailed query data in a structured data format. For example, the third user prompt (802) may be configured to generate a data structure in the form of a list containing multiple detailed query data, and each detailed query data may include an identifier (id), content (text), and an attribute (strict) indicating whether the detailed query data must be satisfied. At this time, the attribute may be set to a value indicating whether the detailed query data is a mandatory condition for determining whether the answer data (630) is correct. For example, if it is a condition that must be satisfied, such as a condition that a specific object type must be included, it may be set to the first attribute (Ex: true), and if it corresponds to an auxiliary condition, it may be set to the second attribute (Ex: false). However, the configuration and output format of the third user prompt (802) are not limited to the above examples and can be implemented in various forms of structured data representation methods.
[0335] The standard large language model (160) can generate at least one detailed query data (810) corresponding to the learning query (600) based on the third prompt (800).
[0336] Additionally, as illustrated in FIG. 8c, the learning unit (140) can generate at least one detailed answer data (830) from the answer data (630) using a reference large language model (160). Here, the detailed answer data is generated by breaking down the contents included in the answer data into a plurality of unit information, and may mean data in which the answer data is expressed as a minimum answer unit.
[0337] For example, let us assume that the learning query (600) is "When and where was the Eiffel Tower built, and what is its purpose?" and the answer data (630) for the learning query (600) is "The Eiffel Tower was built in London, England in 1889 and was created for the World's Fair to commemorate the 100th anniversary of the French Revolution." In this case, the detailed answer data (830) may include answer data that breaks down the content included in the answer data into minimum answer units, such as "The Eiffel Tower was built in 1889," "The Eiffel Tower was built in London, England," "The Eiffel Tower was created to commemorate the 100th anniversary of the French Revolution," and "The Eiffel Tower was built for the World's Fair." As such, as illustrated in (d) of FIG. 10a, the detailed answer data (830) may include first detailed answer data, second detailed answer data, etc.
[0338] Specifically, the learning unit (140) may process as input to the reference large language model (160) a prompt (or a fourth prompt, 820) that includes at least one instruction requesting to generate (or extract) at least one detailed answer data (830) from the answer data (630). For example, as illustrated in FIG. 8d, the fourth prompt (820) may include a learning query (600), answer data (630) for the learning query (600), and at least one instruction requesting to break down the answer data into at least one detailed answer data (or claim). Specifically, the fourth prompt (820) may include at least one instruction requesting to break down the answer data into verifiable detailed answer data in the smallest unit, to express the detailed answer data in the form of a concise sentence, and to restrict the addition of new information not included in the answer data.
[0339] Additionally, the fourth prompt (820) may include a request to generate an attribute indicating whether the detailed answer data is a fact verifiable through external evidence, and may be configured to indicate whether verifiable factual information requiring evidence verification, such as name, date, quantity, relationship, etc. Furthermore, if the answer data consists of a single entity name or a brief expression, the fourth prompt (820) may include at least one instruction requesting to derive the detailed answer data as the minimum answer unit that the answer data signifies based on the relationship with the learning query (600). However, the composition and content of the instruction included in the fourth prompt (820) are not limited to the above examples and may be implemented to include various forms of answer decomposition and factual information extraction methods.
[0340] Furthermore, as illustrated in FIG. 9a, the learning unit (140) can generate second evaluation data (910) for answer data based on at least some of the learning query, answer data (630), detailed query data (810), detailed answer data (830), and reasoning basis data (660) using a reference large language model (160). Here, the second evaluation data (910) may refer to data indicating the result of determining whether an answer for each detailed query data is derived from the answer data and whether the derived detailed answer data corresponds to the correct answer. For example, the second evaluation data may include a second state label (911) related to whether the detailed answer data corresponding to the detailed query data was generated based on information included in the answer data, and a third state label (912) related to whether the answer data includes correct answer data for the entire detailed query data.
[0341] The learning unit (140) can determine a second state label corresponding to each of the at least one detailed query data by using the standard large language model (160) and, based on the thought basis data (660), determining whether the answer data (630) for the learning query (600) includes an answer for each of the at least one detailed query data (810). Furthermore, the learning unit (140) can determine a third state label corresponding to the entire at least one detailed query data (810) by using the standard large language model (160) and, based on the thought basis data (660), determining whether the answer data (630) for the learning query (600) is the correct answer for the entire at least one detailed query data (810).
[0342] Specifically, the learning unit (140) can process as input to the reference large language model (160) a prompt (or a fifth prompt, 900) that includes at least one instruction requesting to specify a second state label corresponding to each of at least one detailed query data among a plurality of preset state labels (or, query matching state labels). Additionally, the learning unit (140) can process as input to the reference large language model (160) a prompt (or a fifth prompt, 900) that includes at least one instruction requesting to specify a third state label corresponding to each of at least one detailed query data among a plurality of preset state labels (or, correct answer state labels).
[0343] For example, as illustrated in (a) of FIG. 9b, the fifth prompt (900) may include at least one instruction requesting a second status label corresponding to each of at least one of at least one of the multiple query matching status labels (e.g., "satisfied", "unsatisfied", "partially_satisfied", "unknown", etc.) by determining whether each of at least one detailed query data (810) contains an answer. (or, fifth system prompt, 901).
[0344] Specifically, the fifth system prompt (901) may include a request to determine whether the answer data satisfies the detailed query data for each of at least one detailed query data (810). At this time, the fifth system prompt (901) may include at least one instruction requesting to determine that the answer data (630) does not satisfy the detailed query data if the answer data (630) does not reflect the specific detailed query data at all. Additionally, the fifth system prompt (901) may include at least one instruction requesting to set a second state label corresponding to a state where judgment is impossible if the answer data (630) reflects the detailed query data but is not supported by the accident basis data (660).
[0345] Additionally, the fifth system prompt (901) may include a request to select one of the plurality of query matching status labels according to the extent that the answer data (630) satisfies the detailed query data. For example, the fifth system prompt (901) may include at least one instruction requesting that a first label (or, first matching label, Ex: "satisfied") be selected if all items included in the detailed query data are satisfied, a second label (or, second matching label, Ex: "partially_satisfied") if only some are satisfied, a third label (or, third matching label, Ex: "unsatisfied") if not satisfied, and a fourth label (or, fourth matching label, Ex: "unknown") if it is difficult to determine.
[0346] Furthermore, the fifth system prompt (901) may include an instruction requesting that identification information of the accident basis data used to determine the second state label be output together, thereby allowing the basis for determining the second state label for each of at least one detailed query data to be clearly presented.
[0347] Additionally, the fifth prompt (900) may include a system prompt (or a fifth system prompt, 901) that includes at least one instruction requesting the identification of a third state label corresponding to at least one detailed query data (810) among a plurality of correct state labels (e.g., "correct", "incorrect", "partially_correct", "unknown", etc.) by determining whether the answer data (630) for the learning query (600) is correct for at least one detailed query data (810).
[0348] However, the configuration and content of the instructions of the above-mentioned fifth system prompt (901) are not limited to the above examples and may be implemented to include various forms of answer verification criteria and judgment methods.
[0349] Meanwhile, the fifth prompt (900) may include a user prompt (or, the fifth user prompt, 902) comprising at least one of a learning query (600), answer data for the learning query (630), detailed query data (810), and reasoning data (660).
[0350] In the reference large language model (160), at least one of a second state label and a third state label corresponding to at least one detailed query data can be specified (or selected, determined) based on the thought basis data. Furthermore, the reference large language model (160) can generate second evaluation data (910) using at least one of the second state label and the third state label corresponding to each of the at least one detailed query data. Furthermore, the learning unit (140) can obtain the second evaluation data (910) from the reference large language model (160).
[0351] Furthermore, as illustrated in FIG. 9c, the learning unit (140) can generate third evaluation data (930) for the answer data based on at least some of the learning query, answer data (630), detailed query data (810), detailed answer data (830), and reasoning basis data (660) using a reference large language model (160). Here, the third evaluation data (930) may refer to evaluation data related to whether logical consistency is satisfied by the reasoning basis data for each of at least one detailed answer data. In this case, logical consistency may refer to whether the detailed answer data (830) is consistently derived based on the reasoning basis data (660) or whether it has a relationship supported or backed by the reasoning basis data (660). Specifically, logical consistency may include whether the detailed answer data (830) can be reasonably derived without contradicting the reasoning data (660), and may be a concept that encompasses cases where it is sufficiently supported by the reasoning data (660), partially supported, or not supported or contradictory.
[0352] Specifically, the learning unit (140) can specify a fourth state label for each of at least one detailed answer data based on thought basis data using a reference large language model. For example, the learning unit (140) can process as input to the reference large language model (160) a prompt (or a sixth prompt, 920) that includes at least one instruction requesting to specify a fourth state label corresponding to each of at least one detailed answer data among a plurality of preset state labels (or answer matching state labels).
[0353] For example, as illustrated in FIG. 9d, the sixth prompt (920) may include at least one instruction requesting that a fourth status label corresponding to each of at least one of at least one detailed answer data be specified by using the accident basis data (660) to determine the logical consistency of the detailed answer data (e.g., "supported (or, first answer consistency status label)", "contradicted (or, second answer consistency status label)", "unsupported (or, third answer consistency status label)", etc.).
[0354] In the reference large language model (160), based on the sixth prompt (920), a fourth state label corresponding to each of at least one detailed answer data (830) can be specified (or selected, determined). Furthermore, the learning unit (140) can calculate at least one of a second evaluation indicator and a third evaluation indicator corresponding to the answer data for the learning query based on the fourth state label corresponding to each of at least one detailed answer data.
[0355] Specifically, the learning unit (140) can calculate a second evaluation index corresponding to the answer data (630) for the learning query (600) based on a fourth state label corresponding to each of at least one detailed answer data. As illustrated in (c) of FIG. 10b, the fourth state label is the second label (or second answer matching) among the preset plurality of labels (or answer matching state labels).
[0356] Alternatively, let us assume a case where the fourth state label corresponding to each of at least one detailed answer data is the first label (or, first answer matching state label, Ex: "supported"). In this case, the learning unit (140) can calculate a second evaluation indicator such that the answer data (630) corresponds to the second evaluation label (Ex: "supported") among a plurality of preset evaluation labels (Ex: "contradicted", "supported", "partially_supported", "unknown", "unsupported").
[0357] As another example, let us assume a case where the fourth state label corresponding to each of at least one detailed answer data corresponds only to the first label (or, first answer matching state label, Ex: "supported") and the third label (or, third answer matching state label, Ex: "unsupported"). In this case, the learning unit (140) can calculate a second evaluation indicator such that the answer data (630) corresponds to the third evaluation label (Ex: "partially_supported") among a plurality of preset evaluation labels (Ex: "contradicted", "supported", "partially_supported", "unknown", "unsupported").
[0358] As another example, if there is no detailed answer data of a preset type among at least one detailed answer data, the learning unit (140) can calculate a second evaluation indicator such that the answer data (630) corresponds to a fourth evaluation label (Ex: "unknown") among a plurality of preset evaluation labels (Ex: "contradicted", "supported", "partially_supported", "unknown", "unsupported"). Here, the preset type can be distinguished based on whether the detailed answer data can be judged for logical consistency based on the accident basis data. Specifically, the preset type may include a first type in which logical consistency can be judged and a second type in which logical consistency cannot be judged. For example, the first type detailed answer data may refer to detailed answer data in which the logical consistency can be judged based on the accident basis data, and the second type detailed answer data in which logical consistency is determined by the accident basis data
[0359] The learning unit (140) can calculate a second evaluation indicator such that, if there is no first type detailed answer data among at least one detailed answer data, the answer data (630) corresponds to a fourth evaluation label (Ex: "unknown") among a plurality of preset evaluation labels (Ex: "contradicted", "supported", "partially_supported", "unknown", "unsupported"). Furthermore, if the answer data (630) does not correspond to the first to fourth evaluation labels, the learning unit (140) can calculate a second evaluation indicator such that it corresponds to a fifth evaluation label (Ex: "unsupported").
[0360] Meanwhile, the learning unit (140) can calculate a third evaluation indicator corresponding to the answer data based on the second evaluation indicator corresponding to the answer data and the third state label. Specifically, as shown in (d) of FIG. 10b, the learning unit (140) can calculate a third evaluation indicator corresponding to the first answer evaluation label when the second evaluation indicator corresponds to the first evaluation label. Alternatively, when the second evaluation indicator does not correspond to the first evaluation label, the learning unit (140) can calculate a third state label (912) corresponding to at least one detailed query data generated from the learning query (600) as the third evaluation indicator.
[0361] Furthermore, the learning unit (140) can calculate an answer verification indicator corresponding to the answer data (630) for the above learning query (600) based on the calculated third evaluation indicator. Here, the answer verification indicator may refer to an indicator calculated by comprehensively reflecting whether the answer data (630) satisfies at least one detailed query data included in the learning query and whether it satisfies logical consistency based on the reasoning basis data.
[0362] As illustrated in (e) of FIG. 10b, the answer verification indicator can be defined as a label value corresponding to a label corresponding to the third evaluation indicator. Specifically, the answer verification indicator can be set to have different values depending on the third evaluation indicator calculated in correspondence with the answer data.
[0363] In other words, the answer verification indicator may refer to an indicator representing the accuracy and reliability of the answer data by comprehensively reflecting whether the answer data satisfies the requirements included in the training query and whether it satisfies logical consistency based on the reasoning data.
[0364] The learning unit (140) can generate a third evaluation data (930) for the answer data based on a second evaluation indicator and a third evaluation indicator corresponding to the answer data using a reference large language model. Specifically, the learning unit (140) can calculate a second evaluation indicator corresponding to the answer data using a fourth state label for each of at least one detailed answer data obtained from the reference large language model (160) and a third state label for at least one detailed query data. Furthermore, the learning unit (140) can calculate a third evaluation indicator corresponding to the answer data based on the calculated second evaluation indicator, and calculate an answer verification indicator according to the third evaluation indicator.
[0365] The learning unit (140) can generate third evaluation data (930) including at least one of a fourth state label corresponding to each of at least one detailed answer data, the second evaluation indicator, the third evaluation indicator, and the answer verification indicator.
[0366] In the present invention, the steps of generating the first evaluation data (710), generating the second evaluation data (910), and generating the third evaluation data (930) are described as being performed sequentially and individually, but are not limited thereto. That is, in one embodiment of the present invention, the steps of generating the first evaluation data (710), generating the second evaluation data (910), and generating the third evaluation data (930) are described as being performed sequentially, but in other embodiments, the steps of generating the first evaluation data (710), generating the second evaluation data (910), and generating the third evaluation data (930) may be performed simultaneously, or some of the steps may be performed in an integrated form.
[0367] In addition, although the learning unit (140) in the present invention is described as sequentially inputting the first prompt, the second prompt, the third prompt, the fourth prompt, the fifth prompt, and the sixth prompt to produce a result corresponding to each, it is not limited thereto. That is, in one embodiment of the present invention, a plurality of prompts may be configured in the form of a single integrated prompt and provided collectively as input to a reference large language model (160), and accordingly, the first evaluation data, the second evaluation data, and the third evaluation data may be produced simultaneously or in parallel. Furthermore, depending on the system implementation method or processing purpose, some of the plurality of prompts may be selectively combined or separated and input, and specific prompts may be omitted or performed in an integrated form with other prompts.
[0368] Furthermore, the configuration of the evaluation data generation process and the prompt input to the reference large language model (160) according to the present invention may be changed or combined in various forms within the scope of not departing from the technical concept of the present invention, and is not limited to the specific implementation method described in this specification.
[0369] In the foregoing, the process of obtaining evaluation data for at least one of the thought process data included in the thought structure data and the answer data to the above-mentioned learning query was described. Below, the process of performing an evaluation of the thought structure data based on the evaluation data will be described.
[0370] Next, in the present invention, a step of performing an evaluation of the accident structure data based on the evaluation data may be carried out.
[0371] As illustrated in FIG. 9e, the learning unit (140) can use a reference large language model (160) to produce an evaluation result (940) for thought structure data (610) to include a final evaluation indicator for thought structure data based on at least some of the first evaluation data (710), second evaluation data (910), and third evaluation data (930).
[0372] Specifically, as illustrated in (a) and (b) of FIG. 10c, the learning unit (140) can calculate a fourth evaluation index (GCS) for detailed answer data based on at least some of the first evaluation data (710), the second evaluation data (910), and the third evaluation data (930). Here, the fourth evaluation index (GCS) may mean an index indicating the logical consistency of the detailed answer data for the learning query.
[0373] Specifically, the fourth evaluation indicator is a fourth status label corresponding to each of the detailed answer data, and can be calculated based on the number of detailed answer data assigned a first label (or, first answer matching status label, Ex: supported), the number of detailed answer data assigned a second label (or, second answer matching status label, Ex: contradicted), and the number of detailed answer data assigned a third label (or, third answer matching status label, Ex: unsupported).
[0374] In this case, the first weight may refer to the weight for detailed answer data where the fourth state label is the second label, and the second weight may refer to the weight for detailed answer data where the fourth state label is the third label. Accordingly, the fourth evaluation indicator may be calculated by reflecting the number of detailed answer data where the fourth state label is the first label as an increasing factor, and reflecting the number of detailed answer data where the fourth state label is the second label and the number of detailed answer data where the fourth state label is the third label as decreasing factors.
[0375] As illustrated in (c) and (d) of FIG. 10c, the learning unit (140) can calculate a fifth evaluation indicator (CS) based on at least some of the first evaluation data (710), the second evaluation data (910), and the third evaluation data (930). Specifically, the fifth evaluation indicator can be calculated using a sixth evaluation indicator corresponding to each of at least one detailed query data and a third weight assigned to the detailed query data. Here, the fifth evaluation indicator may mean an indicator indicating whether the answer data satisfies the answer to at least one detailed query data.
[0376] In this case, the sixth evaluation metric may refer to an metric that quantifies the query matching status label of each of at least one detailed query data. Additionally, the third weight may refer to a weight set according to the importance of the j-th detailed query data.
[0377] Furthermore, as illustrated in (e) of FIG. 10c, the learning unit (140) can calculate a final evaluation indicator based on the fourth evaluation indicator (GCS) and the fifth evaluation indicator (CS). Here, the fourth weight may refer to a weight for adjusting the reflection ratio of the fourth evaluation indicator and the fifth evaluation indicator, and the fifth weight may refer to a fifth weight that can be applied when there is an item among the detailed answer data that contradicts the accident basis data. Additionally, the clamp function may refer to a function for limiting the calculated value to a preset range (e.g., -1 or greater, 1 or less).
[0378] Accordingly, the learning unit (140) can calculate a final evaluation indicator by reflecting the logical consistency of the answer data and the degree of satisfaction with the detailed query data together, and this final evaluation indicator can be used as an evaluation result indicating the quality of the thought structure data for the learning query (600).
[0379] FIG. 10d illustrates an embodiment of an algorithm in which a learning unit (140) uses a reference large language model (160) to produce an evaluation result for thought structure data according to the present invention. However, the method for producing an evaluation result for thought structure data according to the present invention is not limited to the algorithm illustrated above, and various modifications and applications are possible. That is, depending on the purpose of implementation, system environment, or processing performance, some steps of the algorithm may be omitted or added, the order of execution may be changed, or they may be performed in parallel. In addition, each step may be distributed and performed by one or more processors or multiple modules, and some steps may be implemented in an integrated or subdivided form.
[0380] In the foregoing, the process of performing an evaluation of thought structure data based on evaluation data was described. Below, the process of training at least one large language model to be trained using the evaluation results of the thought structure data and at least a portion of the thought structure data will be described.
[0381] Meanwhile, the training data set according to the present invention may be configured through a self-checking learning method comprising answer candidate data for at least one training query, an evaluation result for the training query and the answer candidate data, training data generated based on the evaluation result, and a language model trained based on the training data. Specifically, the training data may include answer candidate data in which the evaluation result satisfies a preset condition related to answer quality as preferred sample data, and answer candidate data in which the evaluation result does not satisfy the preset condition as non-preferred sample data. The configuration of such preferred samples and non-preferred samples is as follows:
[0382] In addition, the preferred sample data and non-preferred sample data of the training data according to the present invention can be determined based on at least one status label included in the evaluation result. That is, if the evaluation result includes a status label indicating whether the answer candidate data matches the accident basis data, the answer candidate data corresponding to the matching label can be classified as preferred sample data, and the answer candidate data corresponding to the mismatch label can be classified as non-preferred sample data. Such label-based classification can quantitatively guarantee the quality of the training data and improve the consistency and stability of the training results.
[0383] Furthermore, in the learning evaluation according to the present invention, the evaluation result can be obtained based on thought basis data extracted from thought process data, and the thought basis data can be obtained from at least one external data source. That is, in the same manner as the method of utilizing thought basis data used in the inference stage, a learning signal can be produced in the learning stage based on thought basis data obtained from an external data source. Through this, objective verification based on external information is possible in the learning stage, and the quality of the learning data can be improved.
[0384] Next, in the present invention, a step of training at least one language model (or large language model) to be trained may be performed using the evaluation result of the thought structure data and at least a portion of the thought structure data.
[0385] The learning unit (140) can generate a learning data set for learning at least one large language model (131) to be learned by using a final evaluation indicator, a learning query, and thought structure data corresponding to the learning query. Furthermore, the learning unit (140) can use the learning data set to train the large language model (131) to generate an answer to at least one query based on at least some of the thought structure data corresponding to at least one query received by the large language model (131) and the evaluation results for the thought structure data.
[0386] Specifically, the learning unit (140) can select or assign weights to at least some of the thought structure data and answer data based on the evaluation data and final evaluation indicators included in the learning data set and reflect them in the learning, and can update the parameters of the large language model (131) by increasing the learning contribution for the learning data set with relatively high final evaluation indicators and decreasing the learning contribution for the learning data set with relatively low final evaluation indicators.
[0387] In the present invention, the learning method of the large language model (131) may vary. For example, the learning unit (140) may train the large language model (131) using supervised fine-tuning, reinforcement learning, or a combination thereof, using a final evaluation indicator. In one embodiment, the learning unit (140) may perform supervised learning by selecting at least some of a plurality of learning data sets based on at least one of a final evaluation indicator, an answer verification indicator, and an idea verification indicator. At this time, the learning unit (140) may train the large language model (131) by preferentially selecting a learning data set in which the answer verification indicator is above a preset first threshold and the idea verification indicator is above a preset second threshold.
[0388] In another embodiment, the learning unit (140) can perform reinforcement learning using the final evaluation indicator as a reward signal. Specifically, the learning unit (140) can compare the final evaluation indicators for multiple thought structure data and answer data generated in response to the same query, and classify results with relatively high final evaluation indicators as preferred data and results with relatively low final evaluation indicators as dispreferred data, and use this to update parameters so that the large language model (131) generates higher final evaluation indicators. Here, the parameters of the large language model (131) include at least one of weight values, bias values, embedding parameters, vision encoder parameters, text encoder parameters, patch embedding parameters, a query / key / value (Q / K / V) matrix of a multi-head attention module, linear transformation coefficients of a feedforward network (FFN), layer normalization coefficients, weights of an attention combination module, an output matrix for calculating token generation logit, scale coefficients and offset coefficients after the activation function, a weight matrix of a multimodal combination module combining image features and text features, a position embedding, a segment embedding, a representation vector of a class token and an image patch token, a learning rate of the vision language model, a dropout rate, and a regularization coefficient. It can mean all learnable parameters.
[0389] Additionally, the learning unit (140) may construct a multi-objective function by combining an error verification indicator, an answer verification indicator, and a final evaluation indicator, and perform training of the large language model (131) based thereon. For example, the learning unit (140) may train the large language model (131) to simultaneously minimize the loss function for accuracy and inference quality by combining a loss function corresponding to the final evaluation indicator, a loss function corresponding to the error verification indicator (or answer verification indicator), and a reward function.
[0390] In another example, the learning unit (140) can provide at least one query included in the learning data set as input to the large language model (131), and can calculate a loss function based on the difference between the output result of the large language model (131) using at least one of thought structure data corresponding to the query, evaluation data corresponding to the thought structure data, and evaluation result, and can adjust the parameters of the large language model by performing a backpropagation algorithm based on the loss function.
[0391] At this time, the learning unit (140) can set weights by reflecting at least one of the thought verification indicator, the answer verification indicator, and the final evaluation indicator when calculating the loss function. For example, the large language model (131) can be trained to consider not only the accuracy of the answer but also the validity and evidence-basedness of the thought process simultaneously by decreasing the loss value or increasing the weight for training data with a relatively high final evaluation indicator, and increasing the loss value for training data with a low final evaluation indicator.
[0392] As an example, the learning unit (140) can fine-tune a large language model using a learning data set. In this case, the learning unit (140) can fine-tun the large language model (131) by using a pre-trained large language model as an initial model and further training it using the learning data set to generate thought structure data for at least one query and derive an answer to said at least one query based on an evaluation of said thought structure data.
[0393] As another example, the learning unit (140) can perform instruction-based learning (instruction tuning) using a learning data set. For example, the learning unit (140) can train a large language model (131) by configuring at least one query included in the learning data set as input data in the form of an instruction, and setting thought structure data and answer data corresponding to the query as response data for the instruction. Accordingly, the large language model (131) can be trained to generate a response including thought structure data and answer data for various types of queries.
[0394] In this way, the learning unit (140) can train a large language model (131) using a learning data set that includes at least one of a final evaluation indicator, a learning query, and thought structure data corresponding to the learning query, so that the large language model (131) can be trained to derive an answer to at least one query based on at least some of the thought structure data corresponding to at least one query received from a user terminal and the evaluation results for the thought structure data.
[0395] In addition, the large language model learning method according to the present invention is not limited to a specific learning method or learning procedure and can be performed using various learning algorithms or learning frameworks including backpropagation-based learning, supervised learning, reinforcement learning, fine-tuning, and combinations thereof.
[0396] The large language model learning method according to the present invention has been described in detail above. Hereinafter, with reference to FIGS. 11a to 11d, a method for generating answers using a language model (or a large language model) learned by the language model learning method (or large language model learning method) according to the present invention will be described in detail. FIGS. 11a to 11d are conceptual diagrams for explaining the method for generating answers using a language model according to the present invention.
[0397] Meanwhile, the answer generation method according to the present invention may have a processing entity configuration different from that of the learning stage. Specifically, in the learning stage, a reference large language model (160) for generating thought structure data and a large language model to be learned may exist separately, whereas in the answer generation method (inference stage), a separate reference large language model is not called, and the pre-trained large language model (131) itself may perform all of the generation of thought structure data, extraction of thought basis data, generation of evaluation data, and output of evaluation results. According to one embodiment, the pre-trained large language model (131) may be a model trained to perform both the generation of thought structure data and the evaluation of thought structure data using a learning data set according to the present invention, and accordingly, a configuration may be provided in which a single model performs both answer generation and self-evaluation in the inference stage. This configuration may provide an environment in which evaluation results for thought structure data are output without calling a separate validator model or a separate external evaluation module in the inference stage.
[0398] At least one language model trained in this manner may have a dual control structure configured to perform both an answer generation operation and an answer evaluation operation during the inference phase. According to one embodiment, the dual control structure may provide an environment in which the same language model can perform all of the generation of thought process data, the generation of answer candidate data, the output of thought process evaluation results, and the output of answer candidate evaluation results during the inference phase, through an internal representation learned via thought process data and evaluation results of a training dataset. This correspondence of data flow between the training phase and the inference phase may provide a technical effect that can improve at least one of the stability of the inference result and the generalization performance of the learning quality by ensuring that the evaluation criteria assigned during the training phase are consistently applied during the inference phase.
[0399] In the present invention, a step of receiving at least one query may be performed (S510, see FIG. 5). Then, in the present invention, a step of processing the query as input to at least one large language model that has been pre-trained to generate an answer to the query based on at least some of thought structure data and evaluation results of the thought structure data may be performed (S520, see FIG. 5).
[0400] At this time, at least one query according to the present invention may not be used in a single stage, but may be utilized in multiple stages in subsequent processing stages. According to one embodiment, at least one query may be utilized as input data in the thought process data generation stage, as context data in the answer candidate data generation stage, as evaluation criteria data in the calculation stage of the thought process evaluation result and the answer candidate evaluation result, and as criteria data for the final response determination in the answer generation stage. That is, at least one query according to the present invention may be input data having a multi-stage utilization chain that performs different roles in each subsequent stage.
[0401] Meanwhile, the step in which at least one query is processed as input to a pre-trained large language model may be configured as an initial processing step for sequentially deriving thought process data, answer candidate data, thought process evaluation results, and answer candidate evaluation results within a single inference flow. According to one embodiment, a chained processing structure may be formed in which thought process data is generated in a subsequent step based on the query provided as input in this step, and the generated thought process data is re-entered as input to the same language model to be referenced for the generation of answer candidate data and the output of evaluation results. Additionally, this step may form a look-ahead reference structure to secure in advance the possibility of extracting thought basis data to be utilized in the evaluation step for thought process data and the evaluation step for answer candidate data performed subsequently.
[0402] As illustrated in FIG. 11a, the control unit (130) may receive at least one query (950) from the user terminal (10). At this time, the query may include at least one of multimodal input data consisting of text, images, videos, voice, and combinations thereof. Specifically, the control unit (130) may receive the query through at least one service page provided to the user terminal (10). Here, the service page according to the present invention may refer to at least one user interface provided to provide an answer to the query. For example, the service page may include a search window or an input window for receiving a query from the user, and may include an area for displaying an answer corresponding to the input query in the form of text, images, graphic objects, or a combination thereof.
[0403] The types, configurations, layout structures, and display forms of information included in the service page according to the present invention may be set in various ways and are not limited to specific information configurations or screen configuration methods. For example, the service page may be configured to include an input area for entering a query, an output area for displaying an answer corresponding to the query, and an additional information area for providing additional information related to the answer, but is not limited thereto. Furthermore, the arrangement method, display order, visual representation form, or interface configuration method of the information displayed on the service page may also be implemented in various ways and may be modified or applied within the scope that does not deviate from the technical concept of the present invention.
[0404] Furthermore, the control unit (130) may provide (or process) the received query (950) as input to a pre-trained large language model (131). At this time, the pre-trained large language model (131) may refer to a large language model trained to generate an answer to the query based on at least some of the thought structure data corresponding to the input query and the evaluation results of the thought structure data.
[0405] In the present invention, the large language model (131) is described as being generated using a training data set generated by a large language model training method, but is not limited thereto, and the large language model (131) may include a model that has been trained or generated in various ways. For example, the large language model (131) may include a pre-trained model that has been trained using large-scale text data in advance, and may be a model generated by further fine-tuning the pre-trained model using the training data set according to the present invention.
[0406] Additionally, the large language model (131) may be a model that has been trained from the beginning using a training data set according to the present invention, or it may be a model that has been created by performing additional training or retraining based on a model trained using a different data set.
[0407] Accordingly, the large language model (131) can be interpreted as a concept that includes all types of artificial intelligence models capable of generating an answer to the query based on at least some of the thought structure data corresponding to the query and the evaluation results of the thought structure data, and the present invention is not limited to a specific model structure or learning method.
[0408] Specifically, a large language model (131) operated by at least one processor can generate thought structure data corresponding to at least one query (950). At this time, the thought structure data may include at least one of at least one query (950), thought process data corresponding to said query (950), and answer data for said query (950). Furthermore, the large language model (131) operated by at least one processor can generate evaluation data for at least some of the at least one thought process data and the answer data for said query included in said thought structure data.
[0409] Specifically, the large language model (131) can generate at least one of detailed query data (961) corresponding to at least one query (950), thought basis data (962) corresponding to thought process data corresponding to said query (950), and detailed answer data (963) for answer data for said query (950).
[0410] In the description of the large language model learning method according to the present invention, the process of generating detailed query data, reasoning basis data, and detailed answer data has been explained in detail; therefore, to avoid duplication of explanation, it will be omitted. At this time, the process by which the large language model (131) generates detailed query data, reasoning basis data, and detailed answer data follows the method described in the large language model learning method according to the present invention.
[0411] However, regarding the method of generating answers, the way in which the large language model (131) generates detailed query data (961), accident basis data (962), and detailed answer data (963) is not necessarily limited to a prompt. For example, the large language model (131) may internally generate detailed query data (961) based on a query (950) received from a user terminal (10), or generate accident basis data (962) based on information obtained from an external data source or previously stored data. Additionally, the large language model (131) may generate detailed answer data (963) using an intermediate representation, hidden state information, or a predefined template that is automatically generated in response to the above query (950) or detailed query data (961).
[0412] That is, the large language model (131) can generate detailed query data (961), reasoning data (962), and detailed answer data (963) based on at least one of a query (950), internally generated information corresponding to the query, or input information provided from an external system, without explicit prompt input. Accordingly, the answer generation method according to the present invention should be interpreted as including various forms of data generation methods, such as automatic generation, event-based generation, and generation by internal system processing, in addition to a prompt-based generation method.
[0413] A large language model (131) can generate evaluation data for thought structure data corresponding to at least one query (950) based on at least one of detailed query data (961), thought basis data (962), and detailed answer data (963). Furthermore, based on the evaluation data in the large language model (131), an evaluation of the thought structure data corresponding to the query (950) can be performed to generate an evaluation result for the thought structure data.
[0414] In the present invention, a step of obtaining an answer to the query from a large language model may be performed (S530, see FIG. 5). Furthermore, in the present invention, a step of providing the obtained answer to a user terminal may be performed (S540, see FIG. 5).
[0415] At this time, the answer generation according to the present invention may be performed through a partial regeneration path determined based on at least one of the result of the thought process evaluation and the result of the answer candidate evaluation. According to one embodiment, instead of performing regeneration on the entire area of the answer candidate data, a partial regeneration path may be adopted in which updates are performed only on a portion of the area determined not to match the thought basis data. This partial regeneration path may provide a technical effect of reducing at least one of the amount of computation and response latency in the inference stage compared to a path that regenerates the entire answer candidate data.
[0416] In addition, the answer generation method according to the present invention may have a directional data processing structure in which the information flow leading from thought process data to answer candidate data and the information flow leading from answer candidate data to an answer via an evaluation result are explicitly separated. This separated information flow can provide an environment in which independent verification and modification can be performed for each of the thought process data, answer candidate data, and evaluation result.
[0417] The control unit (130) can obtain an answer (970) to the query (950) generated from a large language model based on the evaluation result of the thought structure data corresponding to the query. Furthermore, the control unit (130) can provide an answer (970) to at least one query to a user terminal. Specifically, as illustrated in FIG. 11c, the control unit (130) can provide an answer (970) to the query to the user terminal through at least one service page (1000). At this time, the service page (1000) is a user interface displayed on the output unit (or display unit) of the user terminal (10), and may mean one or more screen configurations for providing information related to an answer to a query and the generation of an answer. For example, the control unit (130) can provide the answer (970) to a query received through the service page (1000) to the user terminal (10) by displaying it in a form including text, a table, a chart, an image, video, voice, or a combination thereof.
[0418] Specifically, the control unit (130) may provide at least one of the accident structure data and the evaluation result for the accident structure data to at least one service page through the service page (1000) along with the answer (970) to the query. For example, the control unit (130) may provide the answer (970) to at least one query (950), the accident structure data, the accident process data included in the accident structure data, and the evaluation data (971a) for at least one of the answer (970) to the service page (1000).
[0419] Furthermore, the control unit (130) can calculate a reliability score (972) for the answer (970) using the final evaluation indicator included in the evaluation result of the accident structure data. In the present invention, there may be various methods for calculating the reliability score for the answer (970). As an example, the control unit (130) can calculate the reliability score (972) for the answer (970) by quantifying it according to a pre-set scale based on the final evaluation indicator.
[0420] At this time, the preset scale may be configured to determine that the reliability of the answer (970) is higher as the value of the final evaluation indicator is higher, and that the reliability of the answer (970) is lower as the value of the final evaluation indicator is lower. The control unit (130) may convert the final evaluation indicator into a value within a preset specific range (e.g., 0 to 1 or 0 to 100 range) and provide the calculated reliability score (972) to the service page (1000). In addition,
[0421] Meanwhile, as illustrated in FIG. 11d, the control unit (130) may provide a modified answer (973) to the user terminal for the query (950) based on the evaluation result of the accident structure data. Specifically, if the evaluation result of the accident structure data satisfies a preset condition, the control unit (130) may generate a modified answer to the query based on the evaluation result using the large language model (131). Here, the preset condition may mean a condition satisfied by the final evaluation indicator according to the evaluation result being below a preset threshold.
[0422] When the evaluation result for the thought structure data satisfies a preset condition, the control unit (130) can process at least one of a query (950), an answer (970) to the query, and an evaluation result for the thought structure data corresponding to the query as input to the large language model (131). The large language model (131) can generate modified thought structure data corresponding to the query (950) based on the input data. Furthermore, the large language model (131) can generate modified evaluation data for the modified thought structure data and generate a modified evaluation result based on the modified evaluation data. At this time, the large language model (131) can determine whether the modified final evaluation indicator included in the modified evaluation result satisfies the preset condition.
[0423] Furthermore, in the large language model (131), if the final revision evaluation indicator satisfies the aforementioned preset conditions, processing can be performed to generate a revised answer (973) to the aforementioned query (950) based on the aforementioned revision thought structure data, and if the final revision evaluation indicator satisfies the aforementioned preset conditions, processing can be performed to regenerate the aforementioned revision thought structure data or perform an additional correction process.
[0424] Additionally, the large language model (131) can be configured to update the accuracy and reliability of the answer to the query (950) step by step by repeatedly performing the processes of generating, evaluating, and generating answers to thought structure data according to the final evaluation indicator. Furthermore, the control unit (130) can obtain a modified answer (973) from the large language model and provide the obtained modified answer (973) to at least one service page (1000).
[0425] As described above, according to the present invention, an answer can be generated by simultaneously verifying the thought structure and the answer during the process of generating an answer to a query. Through this, an answer can be generated based not only on the result of the answer generation, but also on at least some of the thought structure data throughout the answer generation process and the evaluation results regarding the thought structure data.
[0426] Furthermore, according to the present invention, a multi-stage thought structure and a final answer can be verified and reflected to generate an answer in an Agentic Search Augmentation Generation (Agentic RAG) environment. Through this, each inference step included in the thought process can be evaluated based on whether it aligns with the thought basis data, and for a query involving multiple stages of thought, an answer can be generated based on the result of evaluating the thought flow and the answer together.
[0427] Furthermore, according to the present invention, an answer can be derived by considering the consistency and reliability of the thought structure and results generated throughout the entire operation process of the agent-based AI. Through this, even in an environment involving tool utilization and multi-stage reasoning, the thought flow of the entire execution process can be evaluated together with the thought basis data, thereby providing an environment in which accuracy indicators and evidence consistency indicators for the answer data are calculated together.
[0428] Furthermore, according to the present invention, a large language model can be trained to simultaneously verify the thought structure and the answer during the process of generating an answer to a query. Through this, the large language model can be trained in a manner that considers the consistency of the generation process rather than simply generating a result, and as the evaluation results of the thought process are reflected in the training data, the large language model can be trained to generate an answer based on the thought structure data and the evaluation results.
[0429] Specifically, according to the present invention, a learning data set in which the thought structure and the answer are simultaneously verified can be generated. Through this, learning data reflecting consistency between the generation process and the result can be constructed, and a learning data set selected by at least one of first evaluation data for the thought process data, second evaluation data for the answer data, and third evaluation data can be used for learning.
[0430] Furthermore, according to the present invention, a large language model can be trained by reflecting the evaluation results of multi-stage thought structures and answers performed in an agent-based artificial intelligence environment. Through this, the evaluation results of multi-stage thought structures and answers can be reflected together in the training, thereby providing an environment in which the trained large language model generates answers based on at least some of the thought structure data and the evaluation results of the thought structure data.
[0431] Meanwhile, the present invention described above can be implemented based on a quantum computer. The present invention implemented based on a quantum computer may include a quantum processor and a quantum memory based on qubits, and may include software and hardware interfaces suitable for quantum computation.
[0432] Quantum processors in quantum computers utilize qubits to parallelize and process complex operations that binary bits in classical computers cannot perform, through quantum entanglement and quantum superposition. Quantum processors process data using quantum gates and can provide exponential speed improvements for specific problems.
[0433] The answer generation method and system according to the present invention may be configured not merely to output an answer to a query, but to verify the reliability of the answer through a data flow structure formed between internal and external data sources of the language model, wherein thought process data, answer candidate data, thought basis data, thought process evaluation results, and answer candidate evaluation results. This data flow structure defines a multi-stage information path that is different from a single-stage input-output mapping, and verification and updating of the answer generation result may be performed by forming intermediate state data on the information path.
[0434] Specifically, according to one embodiment of the present invention, a query can be input into the language model through a first information path and converted into thought process data and answer candidate data, the thought process data can be re-input into the language model through a second information path and converted into a thought process evaluation result, and the thought basis data can be involved in the production of the answer candidate evaluation result through a third information path obtained from an external data source. In this way, the answer generation system according to the present invention does not merely define what processing is performed, but by limiting how information is processed along a specific path, it can provide technical effects regarding the reliability and updateability of the answer generation result.
[0435] Furthermore, the data flow structure according to the present invention can achieve at least one of the following technical improvements. First, by forming a verification path mediated by thought-based data, the number of re-inference or answer regeneration operations required to ensure the reliability of the answer generation result can be reduced, and accordingly, the usage of computational resources required in the inference stage of the language model can be reduced. Second, by a structure in which the same language model cyclically processes thought process data, answer candidate data, and evaluation results, a verification pipeline based on a single model can be implemented without the need to additionally configure a separate verification model, and as a result, the complexity of the entire system and the requirement for redundant storage of model parameters can be reduced. Third, by a learning structure based on preferred / non-preferred sample data according to the thought process evaluation results and answer candidate evaluation results, the loss of general language expression ability (so-called catastrophic forgetting) acquired in the pre-training stage of the language model can be suppressed while the quality of the answer can be improved.
[0436] In addition, the answer generation method according to the present invention is configured to be performed on a computing system comprising at least one memory and at least one processor, and at least one operation among acquiring thought process data, calculating the result of the thought process evaluation, acquiring the basis of the thought data, and updating the answer candidate data can be performed in a distributed manner by a plurality of computing resources interconnected through a communication network. For example, in one embodiment of the present invention, the language model is operated by a first computing resource, the search for the external data source is performed by a second computing resource, and the first computing resource and the second computing resource can exchange the basis of the thought data through the communication network. Such a distributed processing structure may be difficult to perform solely through sequential processing by a single computing resource due to the scale of the data to be processed, response time requirements, or the need to separate access rights to the external data source.
[0437] The data flow structure and distributed processing structure described above can be realized by components that define the input-output relationship between thought process data, answer candidate data, thought process evaluation results, answer candidate evaluation results, and thought basis data in the answer generation method, language model learning method, answer provision service system, and non-transient computer-readable storage medium described in this specification. Accordingly, the answer generation method, language model learning method, answer provision service system, and non-transient computer-readable storage medium of the present invention cannot be performed by human cognitive processing or simple abstract information processing, and can provide the technical effect of answer generation with improved reliability by defining the data flow on the computing system.
[0438] Furthermore, according to one embodiment of the present invention, since thought process data and answer candidate data are generated and evaluated by the same language model within the same inference flow in response to at least one query, the consistency of the answer generation process can be ensured without calling a separate external verification model or verification module. This configuration can provide a technical effect of reducing at least one of the number of models called during the inference phase, the number of data transmissions between modules, and the storage space requirements for intermediate state data.
[0439] In addition, an answer generation method according to one embodiment of the present invention may have an information flow structure in which thought process data and answer candidate data are output together from a language model into which a query is input, and the output thought process data and answer candidate data are re-entered as input to the same or different language model to produce a thought process evaluation result and an answer candidate evaluation result, and the evaluation result is fed back to the input path of the answer generation step. This information flow structure may include a feedback path that is distinguished from a conventional answer generation structure that includes only a unidirectional forward inference flow, and may include a control path that determines whether and to what extent the answer candidate data is updated based on the consistency between the thought process data and the answer candidate data.
[0440] In addition, according to one embodiment of the present invention, a control path may be provided in which the updating of answer candidate data is determined based on at least one state label determined based on accident basis data. The control path may be configured such that if the answer candidate data is determined to match the accident process data, the answer candidate data is determined as the answer as is, and only when the answer candidate data is determined not to match the accident process data is the answer candidate data update performed. Accordingly, compared to a processing structure that performs uniform updating or regeneration for all answer candidate data, at least one of the amount of computation performed during inference and the amount of storage for intermediate state data may be reduced.
[0441] Conventional answer generation structures in the technical field disclosed in this specification may have an information flow in which a first model responsible for answer generation and a second model responsible for verification or evaluation of the answer are configured separately, or an external evaluation module is called separately to perform evaluation of the answer. In such conventional information flows, there may be structural limitations in which format mismatches or context loss occur between the intermediate inference representations or thought process data used for answer generation and the data used in the evaluation stage, resulting in parts of the thought process data not being utilized in the evaluation stage or requiring additional data conversion steps for the evaluation results to be fed back to the answer generation stage.
[0442] In contrast, the answer generation structure disclosed herein may have an information flow structure in which at least one of the thought process data, answer candidate data, thought process evaluation result, and answer candidate evaluation result can be processed in a consistent representation format within the same language model, and the thought basis data extracted from the thought process data is utilized for both the generation of the evaluation result and the determination of the update path for the answer candidate data. Through such architectural changes, at least one of a reduction in the number of model calls, a reduction in the data conversion step between modules, and the assurance of learning quality stability throughout the answer generation-validation cycle can be achieved.
[0443] Meanwhile, the present invention described above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a computer-readable medium (or recording medium).
[0444] Furthermore, the present invention described above can be implemented as computer-readable code or instructions on a medium on which a program is recorded. That is, the present invention can be provided in the form of a program.
[0445] Meanwhile, computer-readable media include all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
[0446] Furthermore, the computer-readable medium may be a server or cloud storage that includes a storage and is accessible to an electronic device via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage via wired or wireless communication.
[0447] A computer program may reach the system (100) through various suitable transmission mechanisms. The transmission mechanism may be, for example, a computer-readable storage medium, a computer program product, a memory device, a recording medium such as a CD-ROM or DVD, or a product that tangibly embodies the computer program. The transmission mechanism may be a signal configured to transmit the computer program according to a predetermined protocol through the air or an electrical connection. The system (100) may propagate or transmit the computer program as a computer data signal.
[0448] Furthermore, references to 'computer-readable storage media,' 'computer program products,' 'computer programs embodied in a tangible form,' etc., or to 'controller,' 'computer,' 'processor,' etc., should be understood to include not only computers with various architectures such as single or multi-processor architectures, sequential (Von Neumann) architectures, or parallel architectures, but also specialized circuits such as Field-Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), signal processing units, and other devices. References to computer programs, instructions, code, etc., should be understood to include software for programmable processors or firmware, such as programmable content for hardware devices, whether it is instructions for a processor or configuration settings for a fixed-function device, gate array, or programmable logic device.
[0449] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, namely a CPU (Central Processing Unit), and no special limitations are placed on its type.
[0450] Meanwhile, the above detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.< / information> < / search> < / think>
Claims
In a method for generating answers performed by a computer, A step of receiving at least one query; A step of inputting the above query into at least one pre-trained language model; A step of obtaining thought process data and answer candidate data for the above query from the above language model; A step of inputting the thought process data and the answer candidate data into the above language model; A step of obtaining at least one of a thought process evaluation result for the thought process data and an answer candidate evaluation result for the answer candidate data from the above language model; A step of generating an answer to the query based on at least one of the result of evaluating the thought process and the result of evaluating the answer candidates through the language model; and A method for generating an answer characterized by including the step of providing the generated answer. In paragraph 1, A method for generating answers characterized in that the above-mentioned answer candidate data is generated by the above-mentioned language model based on the above-mentioned thought process data. In paragraph 2, A method for generating an answer characterized in that the above answer is the result of updating the answer candidate data based on at least one of the above thought process evaluation result and the above answer candidate evaluation result. In paragraph 3, The above answer candidate data is updated through the above language model based on the results of the above thought process evaluation, and An answer generation method characterized by generating the above answer through the language model based on the updated answer candidate data and the answer candidate evaluation results. . In paragraph 1, In the step of providing the above answer, A method for generating an answer characterized by providing at least one of the above-mentioned thought process evaluation result and the above-mentioned answer candidate evaluation result together with the above-mentioned answer on at least one service page. In paragraph 5, If at least one of the above-mentioned thought process evaluation results and the above-mentioned answer candidate evaluation results satisfies a preset condition related to answer quality, The answer is updated based on at least one of the result of the thought process evaluation and the result of the answer candidate evaluation through the above language model, and A method for generating an answer characterized by providing an updated answer on the above service page. In paragraph 1, A method for generating an answer characterized in that at least one of the above-mentioned thought process evaluation result and the above-mentioned answer candidate evaluation result is obtained based on thought basis data extracted from the above-mentioned thought process data by the above-mentioned language model. In Paragraph 7, The above accident basis data includes at least one criterion data that serves as an acquisition criterion for at least one of the above accident process evaluation results and the above answer candidate evaluation results, and A method for generating an answer characterized in that the above reference data is obtained from at least one external data source. In paragraph 8, A method for generating answers characterized in that the above external data source includes at least one of a search engine and a knowledge database. In Paragraph 7, At least one of the above-mentioned thought process evaluation result and the above-mentioned answer candidate evaluation result is obtained based on at least one state label determined by the above-mentioned language model based on the above-mentioned thought basis data, and A method for generating an answer, characterized in that the above-mentioned status label includes a plurality of different labels indicating whether the above-mentioned accident basis data matches at least one of the above-mentioned accident process data and the above-mentioned answer candidate data. In Paragraph 10, The above-mentioned multiple different labels are, A first label indicating that the above accident basis data matches at least one of the above accident process data and the above answer candidate data, and The above accident basis data includes a second label indicating that it is inconsistent with at least one of the above accident process data and the above answer candidate data, and If the above status label is the above first label, the above answer includes the above answer candidate data, and A method for generating an answer characterized in that, when the above-mentioned status label is the above-mentioned second label, the above-mentioned answer includes a result in which the above-mentioned answer candidate data is updated based on at least one of the above-mentioned thought process evaluation result and the above-mentioned answer candidate evaluation result. In paragraph 1, A method for generating answers characterized in that each of the above-mentioned thought process evaluation results and the above-mentioned answer candidate evaluation results includes at least one score calculated by the above-mentioned language model. In a method for learning a language model performed by a computer, A step of obtaining answer candidate data for at least one training query; A step of obtaining an evaluation result for the answer candidate data based on the above learning query and the above answer candidate data; A step of generating training data based on the above evaluation results; and A language model learning method characterized by including the step of training at least one language model based on the above-mentioned learning data. In Paragraph 13, A language model learning method characterized by the fact that the above evaluation result is obtained based on the above learning query, the above answer candidate data, and the above thought process data for the above learning query. In Paragraph 13, The above training data is, The above evaluation results include answer candidate data that satisfies preset conditions related to answer quality as preferred sample data, and A language model learning method characterized by including answer candidate data that does not satisfy the above-mentioned preset conditions as non-preferred sample data based on the above-mentioned evaluation results. In paragraph 15, A language model learning method characterized in that the evaluation result includes at least one state label for the answer candidate data, and the preferred sample data and the non-preferred sample data of the training data are determined according to the state label. In Paragraph 14, The above evaluation results are, A language model learning method characterized by being obtained based on thought basis data extracted from the above-mentioned thought process data, wherein the above-mentioned thought basis data is obtained from at least one external data source. In the answer provision service system, At least one memory storing one or more instructions; and It includes at least one processor that interacts with the above at least one memory, and The above at least one processor executes the above one or more instructions, From a language model, obtain an answer to at least one query and at least one evaluation result related to the reliability of said answer, and An answer providing service system characterized by being configured to provide an answer determined based on the above evaluation results on at least one service page. In Paragraph 18, An answer providing service system characterized by providing at least one of a reliability indicator of the answer and accident basis data for the answer, along with the answer, on the above service page. In Paragraph 18, An answer providing service system characterized by providing the answer to the service page when the above evaluation result satisfies preset conditions related to answer quality, and providing an updated result of the answer to the service page when the above evaluation result does not satisfy preset conditions.
