Information processing method, information processing device, and program for controlling interaction with a large-scale language model using an external structure.

Abstract

This invention relates to an information processing method, an information processing device, and a program that intrinsically control dialogue processing using a large-scale language model through an external structure. Conventional conversational AI has faced challenges such as hasty responses based on ambiguous input, responses that conform to incorrect assumptions, and the generation of unfounded information. [Solution] In this invention, a four-layer external structure consisting of a state management layer, an input normalization layer, a logic verification layer, and an output evaluation layer is applied stepwise to the pre-, middle, and post-stages of the inference processing of a large-scale language model, thereby logically controlling the progress of the dialogue. This suppresses erroneous responses and conformist responses, ensuring high reliability and consistency even in long-duration or multi-stage dialogues.

Description

Technical Field

[0001] The present invention relates to a dialogue system using a large language model (LLM), and more particularly to a dialogue control technique that controls the progress of the dialogue between a user and an LLM by an external structure and suppresses incorrect responses, flattering responses, and hasty responses.

Background Art

[0002] In recent years, dialogue-based AI using large language models has been used in a wide range of applications such as search assistance, business assistance, and creative assistance. However, since conventional LLM dialogue systems generate the most probable response probabilistically for the input context, they generate an estimated response without confirmation even when the user input is ambiguous, generate a response that conforms to the user's incorrect premise, generate non-existent information or information without basis, and deviate from the original context and purpose in a long conversation. It had multiple problems such as

[0003] These problems become a serious obstacle in fields that require high accuracy and consistency, such as business use, contracts, design support, education, legal affairs, and medicine.

[0004] In addition, conventionally proposed RAG and rule-based guardrails mainly focus on complementing external knowledge and output limitation, and do not have a mechanism that internally applies a dialogue progress structure as an external structure to the inference process of the LLM itself and controls state management, input normalization, logical verification, and output evaluation step by step and integrally. Therefore, a mechanism for integrally controlling these four elements according to the progress of the dialogue has not been sufficiently established.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

[0006] [Non-Patent Document 1] Traian Rebedea et al., “NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails”, arXiv 2310.10501. [Non-Patent Document 2] Shinn, N. et al., “Reflexion: Language Agents with Verbal Reinforcement Learning”, NeurIPS 2023. [Non-Patent Document 3] Yao, S. et al., “ReAct: Synergizing Reasoning and Acting in Language Models”, ICLR 2023. [Non-Patent Document 4] Wei, J. et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”, NeurIPS 2022. [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] Conventional dialogue systems using large-scale language models generate response sentences probabilistically based on input information. However, they have several problems, including generating responses based on estimations without confirming the ambiguity of user input, generating accommodating responses to incorrect user assumptions without verifying their validity, generating non-existent or unclearly supported information as fact, and deviating from the original dialogue purpose and conditions during long or multi-stage conversations.

[0008] Furthermore, while existing technologies such as RAG and rule-based guardrails allow for the addition of external knowledge and restriction of output content, it has been difficult to intrinsically and incrementally control the dialogue progress, input normalization, response generation logic validity, and output result evaluation as external structures for the LLM inference process itself.

[0009] Therefore, conventional technologies have faced challenges in ensuring sufficient practicality in business applications where high reliability is required, as they cannot simultaneously satisfy the requirements of explicitly managing the progress of the dialogue while controlling the response, correcting ambiguity and logical inconsistencies in the input during the dialogue, and continuously guaranteeing the accuracy and consistency of the response content.

[0010] The present invention has been made in view of the above problems, and aims to provide a dialogue control technology that can suppress erroneous responses, compliant responses, and premature responses by intrinsically applying an external structure to the internal inference processing of LLM and controlling the progress of the dialogue in a stepwise and logical manner. [Means for solving the problem]

[0011] To solve the above problems, the information processing method according to the present invention employs a configuration that intrinsically applies an external structure to the internal inference processing of a large-scale language model and controls the progress of the dialogue in a stepwise and logical manner.

[0012] Specifically, the information processing method according to the present invention employs a configuration that includes at least four processing layers as an external structure applied to dialogue processing: a state management layer that manages the dialogue progress stage, dialogue purpose, established conditions, and current dialogue position as states and controls the next dialogue processing to be executed based on those states; an input normalization layer that detects ambiguity, logical inconsistencies, missing or incorrect preconditions included in user input and suppresses the generation of hasty responses by requesting confirmation or performing correction processing as necessary; a logic verification layer that verifies the existence of factual accuracy, logical consistency, and conformity in the generated response proposals and suppresses the generation of misinformation, conformity responses, and unfounded assertive responses; and an output evaluation layer that evaluates the consistency with the dialogue purpose, appropriateness of expression, and response quality of the final output response and regenerates or corrects it if it does not meet predetermined standards.

[0013] The information processing method according to the present invention is characterized by the intrinsic application of these four processing layers across the pre-processing, intermediate, and post-processing stages of the inference processing of a large-scale language model, thereby realizing not merely external control, but integrated dialogue control with the inference processing.

[0014] Furthermore, the present invention can also be realized as an information processing device for executing the above-described information processing method, and a program for causing a computer to execute the information processing method. [Effects of the Invention]

[0015] According to the present invention, by intrinsically applying an external structure to the internal inference processing of a large-scale language model and adopting a configuration that controls the progress of the dialogue in a stepwise and logical manner, several effects are achieved.

[0016] Firstly, by using an input normalization layer to detect ambiguity in user input and missing preconditions, and then performing verification processing, the generation of hasty responses based on estimations can be effectively suppressed.

[0017] Secondly, the logical verification layer prevents the generation of responses that conform to the user's incorrect assumptions, thereby maintaining the logical soundness of the dialogue.

[0018] Thirdly, by verifying the factuality and logical consistency of the generated response, it is possible to prevent the output of information that does not exist or information with unclear basis as if it were a fact.

[0019] Fourthly, by explicitly managing the progress stage and conditions of the dialogue through the state management layer, it is possible to suppress deviation from the initial purpose and conditions even in a long or multi-stage dialogue.

[0020] Fifthly, since the output evaluation layer can ensure the quality of the final response, highly reliable dialogue control that can withstand practical use can be realized even in fields such as business support, design support, education, legal affairs, and medical care where high accuracy and consistency are required.

Brief Description of the Drawings

[0021] [Figure 1] It is a diagram showing an example of a dialogue processing flow based on a four-layer external structure according to the present invention. The state management layer and the input normalization layer verify the user input before the inference process, and if an uncertain condition or non-conformance is detected, a confirmation question is generated and the inference process is not executed. When the conditions are determined and the input is conformant, the inference process by the large language model is executed. The logical verification layer and the output evaluation layer verify the inference result, and when there is non-conformance, regeneration or step-by-step rollback is performed by returning to the state management layer.

Modes for Carrying Out the Invention

[0022] Hereinafter, embodiments of the dialogue control technology using the four-layer external structure will be described.

[0023] The dialogue control method according to the present invention operates by intrinsically applying an external structure to the inference processing of a large-scale language model, and specifically consists of four layers: a state management layer that manages the dialogue stage, established information and the current status; an input normalization layer that detects ambiguity, missing premises or logical inconsistencies in user input and verifies or corrects them as necessary; a logic verification layer that verifies the factual accuracy, logical validity and conformity of the generated response to suppress erroneous responses; and an output evaluation layer that evaluates whether the final response conforms to the purpose and quality standards.

[0024] Each processing layer branches its processing based on at least a predetermined judgment target. If the judgment result is unsuitable, processing returns to the layer in question or the preceding layer, and one of the following actions is performed: generating a confirmation request, issuing a regeneration instruction, or stopping the dialogue.

[0025] In an example of the processing flow shown in Figure 1, the four-layer external structure according to the present invention operates as follows.

[0026] The state management layer determines the next possible dialogue stage based on the dialogue stage, confirmed conditions, unconfirmed conditions, and current dialogue position. If the necessary information is unconfirmed, it does not proceed to the next stage and instructs the input normalization layer to generate a confirmation request.

[0027] The input normalization layer determines whether the user input contains ambiguous expressions, missing preconditions, or logical inconsistencies, and generates input correction or confirmation requests based on that determination. If the determination result is insufficient, the inference process is not executed, and the dialogue is stopped.

[0028] The logic verification layer determines the factual accuracy, logical consistency, and presence of accommodating reasoning in the candidate responses to be generated. If an accommodating response or logical inconsistency is detected, the response is discarded and regeneration is instructed.

[0029] The output evaluation layer determines whether the final response conforms to the dialogue objective, definitive conditions, and predetermined quality standards. If it is determined to be non-conforming, it performs regeneration or rolls back to the dialogue stage.

[0030] These four layers of external structure are sequentially applied to the execution process of inference processing by the large-scale language model, allowing for gradual and logical control of the dialogue progression.

[0031] In this specification, "internal application of an external structure" means introducing a four-tier structure, defined independently of the large-scale language model, into the preceding, intermediate, and succeeding stages of the model's inference process, and making it operate in a way that influences the inference process.

[0032] One embodiment of the present invention, FD1 (Free-Dialogue Safety Architecture), is a configuration that maintains natural conversation while suppressing ambiguity, false assumptions, conformity, and hallucinations in user input during free dialogue. In FD1, the state management layer maintains the main topic, purpose, and emotional tone of the dialogue; the input normalization layer does not answer ambiguous questions but generates confirmation questions; the logic verification layer performs conformity checks and factual verification; and the output evaluation layer shapes the response into a natural and consistent one. As a result, misinformation and conformity can be suppressed and logical consistency can be maintained even in free dialogue.

[0033] Another embodiment of the present invention, SD1 (Static Stage-Based Architecture), is a method applied to stage-based tasks such as travel planning, property searching, and business interviews, and proceeds with dialogue according to predefined stages (Init → Ask → Confirm → Generate → Review). In SD1, a state management layer manages roles at each stage, an input normalization layer corrects omissions and inconsistencies in responses, a logic verification layer detects task deviations and inconsistencies, and an output evaluation layer guarantees the quality of the final output. This prevents omissions in interviews and structural deviations, enabling consistent task execution.

[0034] Another embodiment of the present invention, DX1 (Dynamic Exploratory Architecture), is a method for narrowing down objectives by autonomously generating questions in response to ambiguous user requests and exploratory thinking. In DX1, the state management layer dynamically transitions between the exploration phase and the convergence phase, the input normalization layer generates questions that expand the exploration when information is insufficient, the logic verification layer suppresses logical leaps and overgeneralization, and the output evaluation layer evaluates the accuracy of convergence to the objective. This makes it possible to clarify goals step by step, even in tasks with ambiguous requirements.

[0035] Another embodiment of the present invention, TD1 (Thought-Design Architecture), is a method that divides the thought process itself into an external structure to support design thinking and structured thinking. In TD1, the state management layer manages the thinking stages (analysis, abstraction, and reconstruction), the input normalization layer automatically detects leaps in logic, causal misinterpretations, and logical deficiencies, the logic verification layer confirms the validity of inferences, supporting evidence, and consistency of reasoning, and the output evaluation layer verifies the appropriateness of the thought structure. This structurally supports the organization of thoughts and improves the accuracy of idea generation and document structuring.

[0036] Furthermore, another embodiment of the present invention, SD4 (Temporal Spec-Based Architecture), is a method that integrates time axis management into the tiered structure of SD1. In SD4, the state management layer tracks the time state of tasks (Init, Active, Waiting, Updating, and Done), the input normalization layer ensures that an Update phase is always passed when a condition change occurs, the logic verification layer verifies time consistency, including inconsistencies between the past and present, and the output evaluation layer evaluates the stability and reproducibility of task progress. This enables highly stable and reproducible interactive control even in task management and project management where specification changes occur frequently. Note that FD1, SD1, DX1, TD1, and SD4 are examples of interactive control methods utilizing the external structure according to the present invention, and are not limited to these. The present invention is also applicable to other interactive methods that apply a similar four-layer external structure.

Claims

1. An information processing method for controlling dialogue processing by a large-scale language model, which is executed by an information processing device, wherein the information processing device internally applies an external structure to the execution process of inference processing by the large-scale language model in the dialogue processing, thereby controlling the progress of the dialogue in a stepwise and logical manner, wherein the external structure includes at least four layers: a state management layer for managing the state of the dialogue, an input normalization layer for normalizing user input, a logic verification layer for verifying the logical validity of the generated response, and an output evaluation layer for evaluating the quality of the final response, and the information processing device applies the four layers across the preceding, intermediate, and succeeding stages of the inference processing of the large-scale language model.

2. An information processing method according to claim 1, characterized in that the input normalization layer generates a confirmation request to suppress the generation of a hasty response when the user input contains ambiguity or deficiencies in preconditions.

3. An information processing method according to claim 1, characterized in that the logic verification layer detects a conforming response to an incorrect premise of the user and suppresses the generation of said response.

4. An information processing method according to claim 1, characterized in that the state management layer manages the dialogue progress state according to a fixed-stage dialogue flow and controls the acquisition and confirmation of necessary information at each stage.

5. An information processing method according to claim 1, characterized in that the state management layer manages a dynamic dialogue state including a search phase and a convergence phase, and the input normalization layer generates additional questions when there is insufficient information.

6. An information processing method according to claim 1, characterized in that the state management layer manages at least three stages of thought processes: analysis, abstraction, and reconstruction, and the logic verification layer verifies the validity and consistency of the inference.

7. An information processing method according to claim 1, characterized in that the state management layer manages the time state of a task and transitions the dialogue progress to an update phase when a change in conditions is detected.

8. An information processing device that performs dialogue processing using a large-scale language model, comprising a processor that performs the information processing method described in any one of claims 1 to 7.

9. A program that causes a computer to execute the information processing method described in any one of claims 1 to 7.

Images