An active clarification reasoning control method for information missing scenarios

Abstract

The application discloses an active clarification reasoning control method for an information missing scene, and aims to improve the coping ability of a language model when information is insufficient. The method firstly splices a user question and a multi-round historical dialogue into a context state, inputs a self-recurrence language model to be optimized as a strategy model, and trains the model through reinforcement learning. When a key premise is missing, the model actively outputs a clarification request instead of making an unfounded reasoning. An interactive environment provides missing information and feeds back negative or positive rewards, including an unfounded reasoning punishment, a wrong judgment punishment and a solution reward. A comprehensive reward adopts a reinforcement algorithm to optimize model parameters, and finally obtains a question and answer model with an active clarification capability. The application significantly improves premise missing identification and task success rate, reduces invalid long-chain reasoning, reduces reasoning cost, guarantees direct solution when information is complete, enhances the robustness and generalization capability of the model, and is suitable for multiple task domains.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and large-scale language model training, and particularly relates to an active clarification reasoning control method for information-deficient scenarios. Background Technology

[0002] Large-scale language models have been widely used in applications such as mathematical reasoning, question answering, intelligent customer service, and decision support. With the help of existing reinforcement learning training methods, they have demonstrated high accuracy in standard test scenarios with complete information. However, existing evaluation and training processes typically implicitly assume that users provide all the information needed to complete the task in a single input round, allowing the model to directly provide a definite answer. In contrast, in real-world human-computer interaction scenarios, user expressions are often conversational, fragmented, and progressive. Common scenarios include: users gradually supplementing constraints through multiple rounds of dialogue; users only describing the goal without providing key parameters; user-provided conditions being ambiguous or vague; and users omitting crucial boundary conditions or limitations (such as key values, constraints, object attributes, time ranges, billing rules, preference weights, etc.). Under these incomplete input conditions, even models with strong reasoning abilities struggle to obtain verifiable and reproducible correct conclusions without the necessary premises.

[0003] In such scenarios, existing models commonly exhibit an unreliable output pattern: Initially, the model may display prompts such as "more information needed" or express uncertainty, but it then tends to fill in the missing premises and continue generating reasoning. Common methods include defaulting to missing values, assuming unprovided constraints hold, interpreting ambiguity as the most favorable version, or selecting a branch and continuing derivation without user confirmation. The final output is often formally complete and logically consistent, but because the reasoning is based on unverified assumptions, its conclusions lack factual basis and may be inconsistent with the user's true intentions or real constraints, leading to misleading results. This problem directly impacts the system's credibility and usability in scenarios with high accuracy requirements.

[0004] The aforementioned problems can be summarized as the phenomenon of "unfounded reasoning," where the model initiates reasoning and implicitly fills in the missing information even when necessary preconditions are lacking, making the reasoning and conclusions unverifiable. The root cause of this phenomenon is not a lack of reasoning ability on the part of the model, but rather a lack of boundary judgment and control over whether the current input possesses the necessary preconditions for reasoning. Existing reinforcement learning post-training methods typically use "outputting the answer" as the primary optimization objective, lacking explicit training mechanisms and reward constraints for "stopping and requesting clarification when information is insufficient." When training data primarily consists of information-complete, single-round solvable problems, the model is more prone to developing a policy bias of "producing as many answers as possible," leading to a preference for providing an answer rather than actively terminating reasoning, explicitly pointing out missing items, and requesting user supplementation in information-deficient scenarios. Furthermore, even if the model generates uncertainty signals internally, the lack of an effective mechanism makes it difficult to stably transform uncertainty into external "pause and clarification" behavior, resulting in an output trajectory of "first doubt, then fabrication."

[0005] This behavior not only amplifies the risks of illusions and erroneous reasoning but also triggers the generation of invalid, long-chain reasoning, resulting in lengthy responses, increased computational overhead, and reduced interaction efficiency. Therefore, there is an urgent need for a technical solution that can effectively identify missing premises under incomplete input, stop promptly and proactively request clarification when necessary, and reliably solve the problem only after obtaining supplementary information. This would improve the reliability, controllability, and engineering usability of language models in real-world multi-turn interaction scenarios. Summary of the Invention

[0006] The technical problems that this invention aims to solve compared to existing technologies are: existing technologies struggle to identify missing premises; when input information may be incomplete, existing language models lack an effective mechanism to determine whether "necessary premises are complete," leading to continued reasoning and fabrication of missing information under missing premise conditions, resulting in unreliable conclusions; existing technologies lack training signals for "timely cessation and clarification," even if the model identifies information gaps early on, there is a lack of a stable reward mechanism to drive it to immediately stop and raise explicit clarification questions, easily leading to a pattern of "early suspicion, later fabrication"; existing technologies suffer from over-clarification under complete input, if the model becomes overly conservative and frequently requests clarification on complete questions, it will reduce usability and efficiency. To overcome the above-mentioned defects of existing technologies, this invention proposes an active clarification reasoning control method for information-missing scenarios.

[0007] To achieve the above-mentioned objectives, the present invention specifically adopts the following technical solution: In a first aspect, the present invention provides an active clarification reasoning control method for information-deficient scenarios, comprising the following steps: S1: Take the obtained user questions or user instructions as input text, and concatenate the input text and the existing multi-turn historical dialogue text in chronological order to form a context sequence, thus forming the context state of the current round of dialogue; S2: The autoregressive language model to be optimized is used as the policy model. The context state is input into the policy model for training. During training, the policy model first determines whether a key premise is missing based on the context state: if the key premise is not missing, the policy model performs step-by-step reasoning, arrives at the final answer, and outputs the reasoning process text; if the key premise is missing, the policy model actively stops thinking and outputs a text request for clarification and supplementary information to the interactive environment; if the policy model directly outputs the final answer or continues reasoning when the key premise is missing, the interactive environment provides negative feedback for unfounded reasoning as a penalty, and provides negative text prompts to guide the policy model to correct its behavior; if the policy model misjudges that a key premise is missing when it is not missing, the interactive environment provides negative feedback for incorrect judgment as a penalty. S3: After detecting the text request output by the strategy model, the interactive environment provides the missing key premises in the form of feedback and adds them to the historical dialogue text to form the completed context state. S4: Input the completed context state into the policy model for reasoning, and output the final answer and the corresponding reasoning process text; then, compare the final answer with the standard answer corresponding to the input text. If the final answer is consistent with the standard answer, give the reasoning process text a positive solution reward; if the final answer is inconsistent with the standard answer, give the reasoning process text negative feedback. S5: The comprehensive reward is the sum of the solution reward, the detection reward, the negative feedback for incorrect judgments, and the negative feedback for unfounded inferences. Subsequently, a reinforcement learning algorithm was used to refine the policy model parameters. Update to increase the expected total reward. maximize; For strategy models; S6: Use the trained policy model as a question-answering model with proactive clarification and reasoning capabilities. Input user questions or user instructions into the question-answering model and output the final answer.

[0008] Based on the above scheme, each step can be implemented in the following preferred manner.

[0009] As a preferred embodiment of the first aspect above, in step S2, the context state includes at least: the input text of the current round of dialogue, the user supplementary information of the previous rounds of dialogue, and system prompts.

[0010] As a preferred embodiment of the first aspect mentioned above, in step S2, the unfounded reasoning negative feedback... Incorrect judgment negative feedback ;in, and Both represent adjustable parameters, used to indicate the granularity of penalties for unfounded reasoning and incorrect judgments, respectively.

[0011] As a preferred embodiment of the first aspect mentioned above, in step S3, after the strategy model outputs the text request, the interaction environment matches and verifies the text request. If the text request matches a missing premise, the missing key premise is obtained from the constructed dataset. As supplementary information, it is appended to the historical dialogue text to form the completed context state, and the policy model is in the first... A positive reward is given when a test first hits the true missing premise. ;in, To detect the reward coefficient, As the attenuation factor, This is the current dialogue round.

[0012] As a preferred embodiment of the first aspect above, the dataset construction method in step S3 is as follows: locate the key premise sentences from the complete text containing the key premises and delete them to form missing text. The deleted sentences are stored in the backfilling library as missing key premises and aligned with the missing text ID, so that the missing text, the missing key premises and the standard answer constitute a triple to form the dataset.

[0013] As a preferred embodiment of the first aspect mentioned above, in step S4, the strategy model must use two aspects of content for calculation and deduction during the reasoning process to generate the reasoning process text and the final answer; wherein, the first aspect of content is the numerical value and / or constraint in the supplementary information, and the second aspect of content is the reasoning process text output by the strategy model in the historical dialogue.

[0014] As a preferred embodiment of the first aspect mentioned above, in step S4, the reward is calculated. ,in, To solve for the reward coefficient; This represents a binary reward function. When the final answer matches the standard answer, the function outputs a reward of 1; otherwise, the function outputs a reward of 0.

[0015] In a second aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the proactive clarification reasoning control method for information-deficient scenarios as described in any of the solutions in the first aspect above.

[0016] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the proactive clarification reasoning control method for information-deficient scenarios as described in any of the solutions of the first aspect above.

[0017] Fourthly, the present invention provides a computer electronic device, which includes a memory and a processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the proactive clarification reasoning control method for information-deficient scenarios as described in any of the solutions of the first aspect above.

[0018] Compared with the prior art, the present invention has the following advantages: 1. Improved identification of missing premises and task success rate: The model can stop in time when there is insufficient information and raise specific clarification questions, and achieve the correct solution after the information is filled in.

[0019] 2. Reduce unfounded reasoning and reasoning costs: Reduce invalid long-chain reasoning in the absence of premises, and reduce the average generation length and failure rate.

[0020] 3. Balancing complete input efficiency: By eliminating unnecessary clarification penalties, the model tends to solve directly when the information is complete, thus improving usability.

[0021] 4. Enhanced robustness and generalization: In cases where users refuse to provide supplementary information or there is noise in the feedback, it tends to terminate reasonably rather than fabricate information, and can be transferred to different task domains. Attached Figure Description

[0022] Figure 1 This is a flowchart of the steps of the method of the present invention; Figure 2 The graph shows the performance results of the present invention and the basic model on the benchmark dataset; Figure 3 This is a schematic diagram of a computer electronic device provided by the present invention. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0024] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.

[0025] like Figure 1 As shown, in a preferred embodiment of the present invention, the above-mentioned proactive clarification reasoning control method for information-deficient scenarios includes the following steps S1 to S6. The specific implementation process of each step will be described in detail below.

[0026] S1: Take the obtained user question or user instruction as input text, and concatenate the input text with the existing multi-turn historical dialogue text in chronological order to form a context sequence, thus forming the context state of the current round of dialogue. .

[0027] It should be noted that in step S1 of this invention, the context state includes at least: the input text of the current round of dialogue, user supplementary information from previous rounds of dialogue, and system prompts (such as output format constraints). For example, if the model clarifies the user's question / instruction and the user's answer in previous rounds of dialogue, this can also be included in the context state. Furthermore, to facilitate training and discrimination, the text can be standardized, including unit / numerical normalization, symbol cleaning, and annotation of key entities and values, ultimately obtaining a token sequence that can be input into the language model as a context sequence. .

[0028] S2: Use the autoregressive language model to be optimized as the policy model, and the context state The input strategy model is trained. During training, the strategy model first determines whether key premises are missing based on the context state: if key premises are not missing, the strategy model proceeds with step-by-step reasoning, arrives at the final answer, and outputs the reasoning process text; if key premises are missing, the strategy model actively stops thinking and outputs a text request for clarification and supplementary information to the interactive environment, thus avoiding continuing reasoning when information is insufficient; if key premises are missing, and the strategy model directly outputs the final answer or continues reasoning, the interactive environment provides negative feedback for unfounded reasoning. As a penalty, the interactive environment provides negative text prompts to guide the strategy model to correct its behavior; if the strategy model misjudges that a key premise is missing without any missing premises, the interactive environment provides negative feedback for the incorrect judgment. As a penalty, to prevent the policy model from collapsing; where... and Both represent adjustable parameters, used to indicate the granularity of penalties for unfounded reasoning and incorrect judgments, respectively.

[0029] It should be noted that in step S2 of this invention, the input text may be missing text with missing key premises, or it may be complete text containing key premises. For missing text, the model determines whether to continue answering. If the determination is correct, the missing premises are supplemented, and then the model uses the dialogue history and the supplemented information to provide a complete answer. For complete text, the model determines whether to continue solving or with missing key premises, and then the interactive environment determines whether the answer is correct. If correct, positive feedback is given; if incorrect, negative feedback is given.

[0030] It should be noted that although the training scenario in step S2 of this invention is a mathematical scenario, due to the verifiability and uniqueness of the missing conditions in mathematical scenarios, the method of this invention is not limited to this scenario. Similarly, the method of this invention can also be used for missing information scenarios in non-mathematical tasks such as multi-hop question answering, common sense reasoning, and tool call planning.

[0031] In this embodiment, the above scenario is further instantiated: when interacting with a policy model, the user may provide incomplete information. In this case, the policy model faces the problem of insufficient information, which affects its reasoning. For example: Travel Planning: A user requests a strategy model to plan a travel route, but does not specify the destination. The strategy model needs the user to provide information step by step, such as the destination, departure point, and time, in order to generate effective suggestions.

[0032] Mathematical problem: The user provides the mathematical equation C = A + B, and it is known that A = 2, but the value of B is not provided. In this case, the policy model may arbitrarily assume a value for B, leading to invalid reasoning.

[0033] To this end, the present invention designs three active clarification reasoning control methods: Identifying missing information: When the strategy model realizes that key premises (such as the target location or mathematical variable B) are missing, an evaluation should be conducted first, rather than making assumptions directly.

[0034] Proactively stop inference: The policy model should proactively end the current inference process instead of generating a response based on assumptions. This can significantly reduce token consumption, thereby optimizing resource utilization.

[0035] Request clarification: When information is incomplete, the strategy model should proactively request supplementary information from the user. For example, ask for the target location or the specific value of B, in order to facilitate subsequent reasoning.

[0036] S3: After detecting the text request output by the strategy model, the interactive environment provides the missing key premises in the form of feedback and adds them to the historical dialogue text to form the completed context state.

[0037] It should be noted that in step S3 of this invention, after the strategy model outputs the text request, the interactive environment matches and verifies the text request. If the text request matches a missing premise, the missing key premise is obtained from the constructed dataset. As supplementary information, it is added to the historical dialogue text to form the completed context state. And the strategy model in the first A positive reward is given when a test first hits the true missing premise. This encourages the model to detect missing key premises using as few rounds as possible; among them, To detect the reward coefficient, As the attenuation factor, For the current round of dialogue, The smaller the value, the greater the detection reward.

[0038] It should be noted that the dataset construction method in step S3 of this invention is as follows: locating and deleting key premise sentences from complete text containing key premises to form missing text, with the deleted sentences serving as the missing key premises. The data is stored in the backfill library and aligned with the missing text IDs, thus forming a triplet consisting of the missing text, the missing key premise, and the standard answer, forming the dataset.

[0039] In this embodiment, taking a mathematical scenario as an example, the method for constructing the above dataset is as follows: Standard mathematical datasets (such as the GSM8K / MetaMath dataset) are segmented using Python. Considering that key premises in mathematical problems are always given in numerical form, Python is used for number matching of sentences containing numbers. One sentence containing numbers is then randomly deleted. For the remaining sentences, gpt-4o-mini is used as a validator to verify whether the problem is unsolvable, thus forming missing text with missing key premises. The final dataset should contain <missing text, missing key premises, standard answer>.

[0040] In the policy model training of this embodiment, missing text and complete text are mixed in a proportional manner to simultaneously learn the policy boundaries of "missing premise detection and timely stopping" and "correct solution after completion." Unnecessary clarifications on complete text are penalized to control the inference boundary. Furthermore, when the model clarifies and hits the missing item, the interactive environment retrieves the missing text ID. Backfilling is performed to ensure that the interactive supplementary information corresponds one-to-one with the problem instance, thereby achieving a controllable and reproducible training process.

[0041] S4: Complete the context state The input strategy model performs reasoning and outputs the final answer and the corresponding reasoning process text. Then, the final answer is compared with the standard answer corresponding to the input text. If the final answer matches the standard answer, a positive reward is given to the reasoning process text. , To solve for the reward coefficient; This represents a binary reward function. When the final answer matches the standard answer, the function outputs a reward of 1; otherwise, the function outputs a reward of 0. If the final answer does not match the standard answer, negative feedback is given to the reasoning process text.

[0042] It should be noted that in step S4 of this invention, the strategy model must use two aspects for calculation and derivation during the reasoning process to generate the reasoning process text. With the final answer The first aspect is supplementary information. The first aspect consists of numerical values ​​and / or constraints, while the second aspect is the reasoning process text output by the strategy model in historical dialogues.

[0043] It should be noted that in step S4 of this invention, the dialogue terminates after the final answer is generated. If structured output is used, the "reasoning process text" and "final answer" can be output separately to facilitate environment analysis and evaluation, for example... <think> Reasoning process text< / think> <answer> Final Answer< / answer> .

[0044] S5: The comprehensive reward is the sum of the solution reward, the detection reward, the negative feedback for incorrect judgments, and the negative feedback for unfounded inferences. Subsequently, reinforcement learning algorithms (such as PPO) are used to refine the policy model parameters. Update to achieve the expected return. maximize.

[0045] It should be noted that in step S5 of this invention, without changing the interactive training mechanism and the stage reward design, reinforcement learning algorithms such as PPO, GRPO, DAPO, and GSPO can be used to update the parameters of the policy model.

[0046] S6: Use the trained policy model as a question-answering model with proactive clarification and reasoning capabilities. Input user questions or user instructions into the question-answering model and output the final answer.

[0047] It should be noted that in step S6 of this invention, the user questions / user instructions are in text form, which may be missing key premises or may contain key premises. Regardless of the form, the question-answering model can provide the final answer, realizing proactive clarification reasoning control in scenarios with missing information.

[0048] The present invention will now demonstrate the application effect of the proactive clarification reasoning control method for information-deficient scenarios described in S1-S6 of the above embodiments on a specific dataset through a specific example, so as to facilitate understanding of the essence of the present invention.

[0049] Example The implementation process of the proactive clarification reasoning control method for information-deficient scenarios adopted in this embodiment is as described above and will not be repeated here. This embodiment conducted experiments on a constructed missing premise evaluation set and the original complete dataset. In the experimental setup for task and data construction, the experiment simulated a user posing a question containing missing premises. This embodiment required the model to identify the missing premises and complete the answer after obtaining the supplementary premises. To construct training and testing data, this embodiment located and deleted key premise sentences from mathematical word problems or reasoning question-and-answer datasets, generating missing text, while retaining the deleted sentences as supplementary information. The training set consisted of a mixture of missing and complete texts, and reinforcement learning was used to optimize the policy model using the fill-in and reward provided by the interactive environment. In model performance evaluation, this embodiment statistically analyzed metrics such as missing premise detection rate and success rate (SR), and verified the model's performance retention on the complete dataset. The experimental results are shown in Table 1, which demonstrate that the method of this invention significantly improves the ability to identify incomplete inputs and solve the final problem across all model scales. Specifically, on the Qwen2.5-1.5B model, compared to the Base version, the success rate of this invention increased from 1.8% to 61.6%, a 34-fold improvement; simultaneously, the premise detection rate increased from 4.6% to 90.8%. This gain remains significant in larger-scale models: for Qwen2.5-3B, this invention achieved a success rate of 73.5%, superior to the 62.7% of supervised fine-tuning (SFT); for Qwen3-1.7B, this invention achieved a success rate of 72.8%, with a detection rate as high as 96.5%. This verifies the effectiveness of multi-round reinforcement learning in learning from feedback.

[0050] Table 1. Effects of the method of the present invention on GSM8K-Insufficient and MetaMath-Insufficient standards. Furthermore, to verify whether training for premise detection would impair the model's reasoning ability on well-structured inputs (i.e., problems with complete information), this embodiment was evaluated on the publicly available standard inference benchmark datasets GSM8K and MATH500. The results are as follows: Figure 2As shown in the figure. Experimental results demonstrate that the method of this invention not only retains the model's fundamental capabilities in completeness problems but also further enhances its standard inference performance. In tests on the GSM8K dataset, the performance of the Qwen2.5-1.5B model improved from 52.0% to 69.7% (an improvement of 17.7 percentage points), and the Qwen2.5-3B model increased from 77.1% to 83.7%. In the more challenging MATH500 benchmark test, this performance gain was equally significant: the 1.5B model improved from 26.4% to 34.0%, and the 3B model improved from 43.7% to 46.2%.

[0051] It is understood that the proactive clarification reasoning control method for information-deficient scenarios described in S1-S6 above can essentially be implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the proactive clarification reasoning control method for information-deficient scenarios provided in the above embodiments. This product includes a computer program / instruction, which, when executed by a processor, can implement the proactive clarification reasoning control method for information-deficient scenarios as described in the above embodiments.

[0052] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the active clarification reasoning control method for information-deficient scenarios provided in the above embodiments, such as... Figure 3 As shown, it includes a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the proactive clarification reasoning control method for information-deficient scenarios in the above embodiments when executing the computer program.

[0053] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0054] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the active clarification reasoning control method for information-deficient scenarios provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can realize the active clarification reasoning control method for information-deficient scenarios in the above embodiments.

[0055] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by a processor, which can perform the aforementioned steps S1 to S6.

[0056] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0057] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0058] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A proactive clarification reasoning control method for information-deficient scenarios, characterized in that, Includes the following steps: S1: Take the obtained user questions or user instructions as input text, and concatenate the input text and the existing multi-turn historical dialogue text in chronological order to form a context sequence, thus forming the context state of the current round of dialogue; S2: The autoregressive language model to be optimized is used as the policy model. The context state is input into the policy model for training. During training, the policy model first determines whether key premises are missing based on the context state: if key premises are not missing, the policy model performs step-by-step reasoning, arrives at the final answer, and outputs the reasoning process text; if key premises are missing, the policy model actively stops thinking and outputs a text request for clarification and supplementary information to the interactive environment; if the policy model directly outputs the final answer or continues reasoning when key premises are missing, the interactive environment provides negative feedback for unfounded reasoning as a penalty, and provides negative text prompts to guide the policy model to correct its behavior. If the strategy model misjudges that a key premise is missing without any key premise being missing, the interaction environment will provide a negative feedback for the incorrect judgment as a penalty. S3: After detecting the text request output by the strategy model, the interactive environment provides the missing key premises in the form of feedback and adds them to the historical dialogue text to form the completed context state. S4: Input the completed context state into the policy model for reasoning, and output the final answer and the corresponding reasoning process text; Next, the final answer is compared with the standard answer corresponding to the input text. If the final answer matches the standard answer, a positive reward is given to the reasoning process text; if the final answer does not match the standard answer, negative feedback is given to the reasoning process text. S5: The solution reward, detection reward, negative feedback for incorrect judgment, and negative feedback for unfounded reasoning are added together as the comprehensive reward. Then, a reinforcement learning algorithm is used to update the policy model parameters to maximize the expected value of the comprehensive reward. S6: Use the trained policy model as a question-answering model with proactive clarification and reasoning capabilities. Input user questions or user instructions into the question-answering model and output the final answer.

2. The proactive clarification reasoning control method for information-deficient scenarios as described in claim 1, characterized in that, In step S2, the context state includes at least: the input text of the current round of dialogue, the user supplementary information of the previous rounds of dialogue, and system prompts.

3. The proactive clarification reasoning control method for information-deficient scenarios as described in claim 1, characterized in that, In step S2, negative feedback of unfounded reasoning Incorrect judgment negative feedback ;in, and Both represent adjustable parameters, used to indicate the granularity of penalties for unfounded reasoning and incorrect judgments, respectively.

4. The proactive clarification reasoning control method for information-deficient scenarios as described in claim 1, characterized in that, In step S3, after the strategy model outputs a text request, the interactive environment matches and validates the text request. If the text request matches a missing premise, the missing key premise is retrieved from the constructed dataset. As supplementary information, it is appended to the historical dialogue text to form the completed context state, and the policy model is in the first... A positive reward is given when a test first hits the true missing premise. ;in, To detect the reward coefficient, As the attenuation factor, This is the current dialogue round.

5. The proactive clarification reasoning control method for information-deficient scenarios as described in claim 4, characterized in that, The dataset construction method in step S3 is as follows: locate the key premise sentences from the complete text containing the key premises and delete them to form missing text. The deleted sentences are stored in the backfilling library as missing key premises and aligned with the missing text ID. Thus, the missing text, the missing key premises and the standard answer constitute a triple to form the dataset.

6. The proactive clarification reasoning control method for information-deficient scenarios as described in claim 1, characterized in that, In step S4, the strategy model must use two aspects of content for calculation and deduction during the reasoning process to generate the reasoning process text and the final answer; the first aspect is the numerical values ​​and / or constraints in the supplementary information, and the second aspect is the reasoning process text output by the strategy model in the historical dialogue.

7. The proactive clarification reasoning control method for information-deficient scenarios as described in claim 1, characterized in that, In step S4, the reward is calculated. ,in, To solve for the reward coefficient; This represents a binary reward function. When the final answer matches the standard answer, the function outputs a reward of 1; otherwise, the function outputs a reward of 0.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it can implement the proactive clarification reasoning control method for information-deficient scenarios as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the proactive clarification reasoning control method for information-deficient scenarios as described in any one of claims 1 to 7.

10. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement, when executing the computer program, the proactive clarification reasoning control method for information-deficient scenarios as described in any one of claims 1 to 7.

Images