Method and system for dynamic search decision scheme determination based on monte carlo tree search

Abstract

This invention belongs to the fields of artificial intelligence and natural language processing. To address the problems of redundant retrieval and the inability to perform multi-step reasoning and dynamic programming in traditional retrieval enhancement generation methods, it provides a method and system for determining dynamic retrieval decision schemes based on Monte Carlo tree search. The method includes receiving an initial query question as the starting point of the reasoning process to trigger the invocation of a large language model; invoking the large language model to perform subquery generation and action selection operations; iteratively exploring the reasoning actions of the subquery question based on Monte Carlo tree search; and dynamically filtering the reasoning path by considering the confidence score and honesty score of the large language model to obtain a complete set of reasoning trajectories; constructing preference pairs for each subquery question based on the complete set of reasoning trajectories to optimize the reasoning retrieval strategy, enabling the large language model to autonomously determine the optimal retrieval time. This reduces invalid retrieval and multi-step reasoning and dynamic programming.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and natural language processing, and particularly relates to a method and system for determining dynamic retrieval decision schemes based on Monte Carlo tree search. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Traditional Retrieval-augmented Generation (RAG) methods, when applied to fields such as intelligent question answering and knowledge retrieval, primarily rely on the static combination of large language models and external knowledge bases. Typical retrieval schemes include:

[0004] Option 1: The serial retrieval generation scheme forces the retrieval module to be called in each round of reasoning in the large language model. This results in a strong correlation between the number of retrievals and the complexity of the problem, and high computational cost.

[0005] Option 2: Rule-driven retrieval generation solution, which calls the retrieval module based on keyword matching or fixed triggering conditions (such as confidence threshold), but still has the problem of difficulty in handling fuzzy queries or requiring multiple steps of reasoning;

[0006] Option 3: End-to-end joint training scheme, which jointly optimizes the large language model and retrieval module, but lacks control over intermediate reasoning paths, and is prone to failure in multi-hop reasoning due to the accumulation of incorrect intermediate answers.

[0007] In summary, traditional Retrieval-augmented Generation (RAG) methods suffer from low knowledge retrieval accuracy and incomplete information capture. Fixed retrieval timing and frequency can lead to redundant retrieval, and they also lack dynamic programming for multi-step reasoning in complex problems. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a method and system for determining dynamic retrieval decision schemes based on Monte Carlo tree search. This method can formalize the reasoning retrieval process into a Markov decision process and support flexible action combinations. Through Monte Carlo tree search and reasoning path preference learning mechanisms, it achieves deep coordination between retrieval and reasoning, reduces invalid retrieval, and improves the computational efficiency of the model and the accuracy of answering complex questions.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] The first aspect of the present invention provides a method for determining dynamic retrieval decision schemes based on Monte Carlo tree search.

[0011] In one or more embodiments, a method for determining a dynamic retrieval decision scheme based on Monte Carlo tree search is provided, including:

[0012] The system receives the initial query question and uses it as the starting point for the reasoning process, triggering the invocation of the large language model operation; the reasoning process is pre-constructed as a Markov decision process.

[0013] The large language model is invoked to perform subquery generation and action selection operations. The reasoning actions of the subquery problem are explored iteratively based on Monte Carlo tree search. At the same time, the confidence score and honesty score of the large language model are considered to dynamically filter the reasoning path and obtain a complete set of reasoning trajectories.

[0014] Based on the complete set of reasoning trajectories, preference pairs are constructed for each sub-query question, thereby optimizing the reasoning path to enable the large language model to autonomously determine the best retrieval time.

[0015] As one implementation method, the process of dynamically selecting inference paths includes:

[0016] For the subquery problem generated iteratively by the large language model, a confidence score of the large language model is generated, and then the corresponding reasoning action is selected and an honesty score of the large language model is generated, resulting in the corresponding reward function.

[0017] During the reasoning process of exploring subquery questions through iterative exploration based on Monte Carlo tree search, the score of each reasoning path is updated through a reward function. After the iteration is completed, reasoning paths with scores higher than a preset threshold are selected.

[0018] As one implementation method, the reasoning action is any one of direct answering, retrieval answering, query transformation, and final answering.

[0019] As one implementation method, if the confidence score of the large language model exceeds the preset confidence score threshold, the user is allowed to choose between a direct answer or a final answer action; otherwise, a retrieval answer or query conversion action is triggered.

[0020] As one implementation method, when the reasoning action is a direct answer or a final answer, the honesty score of the large language model ranges from [value missing]. When the reasoning action is to retrieve an answer or transform a query, the honesty score of the language model is 1.

[0021] In one implementation, the expression for the reward function is obtained by summing the upper confidence interval and the question-answering utility factor of the large language model; the question-answering utility factor of the large language model is the product of the confidence score and honesty score of the large language model and a constant factor.

[0022] A second aspect of the present invention provides a dynamic retrieval decision scheme determination system based on Monte Carlo tree search.

[0023] In one or more embodiments, a dynamic retrieval decision scheme determination system based on Monte Carlo tree search includes:

[0024] The reasoning process start point acquisition module is used to receive the initial query question and use it as the starting point of the reasoning process to trigger the invocation of the large language model operation; the reasoning process is pre-constructed as a Markov decision process.

[0025] The reasoning path dynamic optimization module is used to call the large language model to perform subquery generation and action selection operations. It explores the reasoning actions of the subquery problem iteratively based on Monte Carlo tree search, and dynamically filters the reasoning path by considering the confidence score and honesty score of the large language model to obtain a complete set of reasoning trajectories.

[0026] The optimal retrieval timing autonomous judgment module is used to construct preference pairs for each subquery question based on the complete inference trajectory set, thereby optimizing the inference path and enabling the large language model to autonomously determine the optimal retrieval timing.

[0027] A third aspect of the present invention provides a computer-readable storage medium.

[0028] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the dynamic retrieval decision scheme determination method based on Monte Carlo tree search as described above.

[0029] A fourth aspect of the present invention provides a computer program product.

[0030] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps in the dynamic retrieval decision scheme determination method based on Monte Carlo tree search as described above.

[0031] A fifth aspect of the present invention provides an electronic device.

[0032] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the dynamic retrieval decision scheme determination method based on Monte Carlo tree search as described above.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] (1) This invention constructs the reasoning process as a Markov decision process, quantifies the retrieval decision path, and combines it with Monte Carlo tree search and introduces an evaluation strategy that combines confidence and honesty indicators to dynamically optimize the model reasoning path. This breaks through the traditional fixed retrieval mode, realizes autonomous decision-making on when to call external retrieval, reduces redundant retrieval and lowers computational costs.

[0035] (2) The present invention includes four types of reasoning actions: direct answer, retrieval answer, query transformation and final answer. It covers the collaborative scenarios of parameterized knowledge and external knowledge of large language models, and supports question decomposition and query rewriting, which helps to improve the accuracy of retrieving complex questions.

[0036] (3) This invention utilizes Monte Carlo tree search to effectively explore potential reasoning trajectories. By iteratively constructing and evaluating reasoning paths, it can prioritize paths at a lower retrieval cost, balancing accuracy and computational efficiency. Furthermore, it optimizes reasoning paths based on preferences for each sub-query question, significantly improving the dynamic decision-making and reasoning capabilities generated by the enhanced retrieval of the large language model. This endows the large language model with human-like reasoning flexibility and resource utilization efficiency, making it suitable for knowledge-intensive and complex question-answering scenarios. Attached Figure Description

[0037] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0038] Figure 1 This is a flowchart illustrating the dynamic retrieval decision scheme determination method based on Monte Carlo tree search according to an embodiment of the present invention.

[0039] Figure 2 This is a flowchart illustrating the dynamic retrieval decision-making process of the Monte Carlo tree search-based embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram of the dynamic retrieval decision scheme determination system based on Monte Carlo tree search according to an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0045] Figure 1 This is a flowchart illustrating a dynamic retrieval decision scheme determination method based on Monte Carlo tree search, as described in an embodiment of the present invention. Figure 1 The dynamic retrieval decision scheme determination method based on Monte Carlo tree search in this embodiment may include the following steps S101~S103.

[0046] S101 receives the initial query question and uses it as the starting point for the inference process to trigger the invocation of the large language model operation; the inference process is pre-constructed as a Markov Decision Process (MDP). This achieves the quantification of the retrieval decision path.

[0047] The large language model in this invention refers to a deep learning model trained on a large amount of text data, enabling the model to generate natural language text or understand the meaning of language text. These models can provide in-depth knowledge and language production on various topics by training on massive datasets. The core idea is to learn the patterns and structures of natural language through large-scale unsupervised training, simulating the human language cognition and generation process to some extent. Examples of large language models include GPT, LLaMA, and DeepSeek.

[0048] Specifically, the reasoning and retrieval process of a large language model is defined as a quadruple MDP ( ):

[0049] Representing the state space, each state It describes the reasoning process of a large language model, in which For the initial query question, They represent the first Subquery questions and intermediate answers generated by step-by-step reasoning; .

[0050] Representing the action space, this embodiment of the invention defines four types of reasoning actions, including: direct answer, retrieval answer, query transformation, and final answer. This bridges the gap between large language model reasoning and human cognition in the RAG (Retrieval-augmented Generation) scenario.

[0051] Specifically, direct answer: directly answer the question using the parameter knowledge of the large language model, without relying on any external knowledge;

[0052] Retrieval Response: Retrieve relevant knowledge from external knowledge bases to support subsequent reasoning;

[0053] Query transformation: Transform the query, such as rewriting, back suggestions, and query breakdown;

[0054] Final answer: By combining the historical reasoning process, intermediate answers, and the initial question, the final answer is generated, and the model reasoning stops.

[0055] Represents state transitions, based on state Perform reasoning actions After the operation, the environment will be updated. When reasoning action When answering directly, the intermediate answer Intermediate answers are generated by the model itself; during reasoning actions... When retrieving answers, intermediate answers The model retrieves documents and generates intermediate answers; during reasoning actions... For intermediate answers during query conversion Sub-problems by model Rewrite and use as an intermediate answer; when reasoning action When the final answer is given, the intermediate answer is... The final answer is generated by combining the model with information from the historical reasoning process.

[0056] The reward function, in this embodiment, is obtained by summing the upper confidence interval and the question-answering utility factor of the large language model. The question-answering utility factor of the large language model is the product of the confidence score and honesty score of the large language model and a constant factor, and its specific expression is as follows:

[0057] ;

[0058] The reward function described above consists of two parts;

[0059] This refers to the Upper Confidence Bound Apply to Tree (UCT) algorithm, used to balance exploration and exploitation in Monte Carlo tree search; where, This represents the cumulative reward for the child nodes. This indicates the number of times the child node has been visited. Indicates the number of times the parent node has been visited; It is a constant used to control the trade-off between exploration and exploitation; This represents the logarithmic function with base e.

[0060] This represents the question-answering utility factor of a large language model. Indicates the use of historical states Assess the current issues The model confidence score, with a value range of ; Indicates the evaluation of intermediate answers The model's honesty score.

[0061] When the reasoning action is a direct answer or a final answer The range of values ​​is ;

[0062] When the reasoning action is to retrieve an answer or transform a query. The value of is 1; This represents a constant factor used to balance confidence and honesty in a large language model.

[0063] Compared with existing upper bound confidence interval algorithms, the reward function proposed in this invention not only evaluates the correctness of the final answer and the cost of retrieval, but also focuses on the confidence and honesty of the large language model in answering the question, which helps to reduce the illusion of the large language model while balancing model reasoning and retrieval.

[0064] This invention transforms the dynamic retrieval decision-making process into a quantifiable and optimizable sequential decision-making problem. This method overcomes the limitations of fixed retrieval patterns in traditional retrieval enhancement generation techniques, achieving coordinated optimization of retrieval timing, frequency, and reasoning paths.

[0065] S102, invoke the large language model to perform subquery generation and action selection operations, explore the reasoning actions of the subquery problem iteratively based on Monte Carlo tree search, and dynamically filter the reasoning path by considering the confidence score and honesty score of the large language model to obtain a complete set of reasoning trajectories.

[0066] Specifically, the confidence score and honesty score of the large language model are both in the range of 0 to 1.

[0067] Each complete reasoning trajectory in the complete reasoning trajectory set consists of a subquery question, its corresponding intermediate answer, and reasoning action.

[0068] Monte Carlo Tree Search (MCS) is a heuristic search algorithm based on stochastic simulation, primarily used for optimal strategy selection in complex decision-making scenarios. Its core idea is to gradually construct an asymmetric search tree by repeatedly simulating decision paths, dynamically focusing on high-potential branches to reduce ineffective searches, and ultimately finding the solution with the highest return. The core of Monte Carlo Tree Search generation lies in the action space, which defines the scope of tree exploration.

[0069] In step S102, the process of dynamically selecting inference paths includes:

[0070] S1021, For the subquery problem generated iteratively by the large language model, the confidence score of the large language model is generated accordingly, and then the corresponding reasoning action is selected and the honesty score of the large language model is generated accordingly, and the corresponding reward function is obtained.

[0071] S1022, during the reasoning process of exploring the subquery problem iteratively based on Monte Carlo tree search, the score of each reasoning path is updated through the reward function. After the iteration is completed, the reasoning paths with scores higher than the preset threshold are selected.

[0072] The four types of reasoning actions in this embodiment define a highly diverse action space. The model inference retrieval path is constructed using Monte Carlo tree search, integrating different solution strategies for each subquery in the inference process. Unlike existing methods, this embodiment introduces the concepts of model confidence and honesty during the Monte Carlo tree search process to reduce the illusion of large language models and improve the efficiency of inference path search. Specifically, for a given input question, the large language model generates a series of sub-questions and explores four inference actions; each round of inference action exploration consists of the following three parts:

[0073] First, the large language model utilizes historical states. Assess the current issues Confidence.

[0074] Secondly, when the confidence score is greater than the threshold (The value can be set according to the actual situation) When it is, it means that the large language model is confident in its reasoning for the current question, and the reasoning action can be either a direct answer or a final answer; otherwise, it means that the large language model needs additional information or in-depth analysis for its reasoning for the current question, and the reasoning action can be either a retrieval answer or a query transformation.

[0075] Finally, for both direct and final answers, the honesty of the current question answer is evaluated by the large language model, with a value ranging from [value missing]. For retrieval answers and query transformations, the honesty of the current question answer is set to 1, assuming that the large language model fully trusts the information obtained through retrieval and query transformation.

[0076] Through each iteration, the Monte Carlo tree grows larger, all nodes continuously update their scores based on the reward function, and different reasoning paths are explored.

[0077] S103: Based on the complete set of reasoning trajectories, construct preference pairs for each sub-query question, and then optimize the reasoning path to enable the large language model to autonomously determine the best retrieval time.

[0078] This invention utilizes the Monte Carlo tree search method to efficiently explore potential inference paths. By iteratively constructing and evaluating inference paths, it prioritizes these paths with low retrieval costs, balancing accuracy and computational efficiency. The above process search is performed on a training dataset to obtain an adaptive inference retrieval process for a specific problem, which includes the optimal answer strategy for each subquery question and determines whether retrieval is necessary. Based on the inference retrieval process data, this invention constructs preference pairs for each subquery to indicate the preferred action selection.

[0079] In some alternative embodiments, existing algorithms such as Group Relative Policy Optimization (GRPO) can be used to optimize the inference path, thereby obtaining an inference retrieval strategy to enable the large language model to autonomously determine the optimal retrieval time. GRPO is a reinforcement learning (RL) algorithm specifically designed to enhance the inference capabilities of large language models (LLMs). GRPO optimizes the model by evaluating mutually related groups of responses. This approach improves training efficiency, making GRPO an ideal choice for inference tasks requiring complex problem-solving and long-chain reasoning.

[0080] It is understood here that in other embodiments, preference pairs may be constructed based on each subquery question, and other existing optimization algorithms may be used to optimize the inference path in order to enable the large language model to autonomously determine the best retrieval time.

[0081] This embodiment uses a preference learning process based on reinforcement learning to enable a large language model to learn when to retrieve external information, thereby maximizing the model's reasoning and retrieval capabilities while reducing unnecessary retrievals.

[0082] like Figure 3 As shown, the dynamic retrieval decision scheme determination system based on Monte Carlo tree search provided in this embodiment of the invention can be implemented in software. The dynamic retrieval decision scheme determination system based on Monte Carlo tree search includes the following software modules:

[0083] The reasoning process starting point acquisition module 301 is used to receive the initial query question and use it as the starting point of the reasoning process to trigger the invocation of the large language model operation; wherein, the reasoning process is pre-constructed as a Markov decision process;

[0084] The reasoning path dynamic optimization module 302 is used to call the large language model to perform subquery generation and action selection operations. It explores the reasoning actions of the subquery problem iteratively based on Monte Carlo tree search, and dynamically filters the reasoning path by considering the confidence score and honesty score of the large language model to obtain a complete set of reasoning trajectories.

[0085] The optimal retrieval timing autonomous judgment module 303 is used to construct preference pairs for each subquery question based on the complete inference trajectory set, thereby optimizing the inference path and enabling the large language model to autonomously judge the optimal retrieval timing.

[0086] It should be noted here that, Figure 3 The dynamic retrieval decision scheme based on Monte Carlo tree search determines the various modules in the system, and... Figure 1 Each step in the dynamic retrieval decision scheme determination method based on Monte Carlo tree search in the article corresponds to the previous one, and their specific implementation process is the same, so it will not be repeated here.

[0087] Figure 4 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of the present invention. It can be understood that... Figure 4 This only shows an exemplary structure of an electronic device, not all structures; implementation is possible as needed. Figure 4 The structure shown may be part or all of the structure.

[0088] An electronic device provided in this embodiment of the invention includes: at least one processor 401, a memory 402, a user interface 403, and at least one network interface 404. A dynamic retrieval decision scheme based on Monte Carlo tree search determines that the various components in the system are coupled together via a bus system 405. It can be understood that the bus system 405 is used to implement communication between these components. In addition to a data bus, the bus system 405 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 4 The various buses are all labeled as bus system 405. Among them, user interface 403 may include display, keyboard, mouse, trackball, click wheel, buttons, touchpad or touch screen, etc.

[0089] It is understood that memory 402 can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The memory 402 in this embodiment of the invention is capable of storing, for example... Figure 1The diagram illustrates the computer programs corresponding to each step in the dynamic retrieval decision-making method based on Monte Carlo tree search. The operating system contains various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application programs can contain various other applications.

[0090] In other embodiments, the dynamic retrieval decision scheme determination system based on Monte Carlo tree search is implemented using a combination of hardware and software. The dynamic retrieval decision scheme determination system based on Monte Carlo tree search can also be directly embodied as a combination of software modules executed by processor 401. The software modules can be located in a storage medium located in memory 402. Processor 401 reads the executable instructions included in the software modules in memory 402 and combines them with necessary hardware (e.g., including processor 401 and other components connected to bus 405) to complete the dynamic retrieval decision scheme determination method based on Monte Carlo tree search provided in the embodiments of the present invention.

[0091] As an example, processor 401 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0092] As an example of the hardware implementation of the dynamic retrieval decision scheme determination system based on Monte Carlo tree search provided in this embodiment of the invention, the device provided in this embodiment of the invention can be directly executed by a processor 401 in the form of a hardware decoding processor. For example, it can be executed by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the dynamic retrieval decision scheme determination method based on Monte Carlo tree search provided in this embodiment of the invention.

[0093] In this embodiment of the invention, memory 402 is used to store various types of data to support the operation of the dynamic retrieval decision scheme determination system based on Monte Carlo tree search. Examples of such data include: any executable instructions for operating on the dynamic retrieval decision scheme determination system based on Monte Carlo tree search, such as executable instructions that can be included in a program implementing the dynamic retrieval decision scheme determination method based on Monte Carlo tree search according to this embodiment of the invention.

[0094] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including functions for executing... Figure 1 The program code for the method shown. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by the central processing unit, it performs the various functions defined in the apparatus of this application.

[0095] in, Figure 1 The computer program instructions corresponding to the method shown may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0096] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining dynamic retrieval decision schemes based on Monte Carlo tree search, characterized in that, include: The system receives the initial query question and uses it as the starting point for the reasoning process, triggering the invocation of the large language model operation; the reasoning process is pre-constructed as a Markov decision process. The large language model is invoked to perform subquery generation and action selection operations. The reasoning actions of the subquery problem are explored iteratively based on Monte Carlo tree search. At the same time, the confidence score and honesty score of the large language model are considered to dynamically filter the reasoning path and obtain a complete set of reasoning trajectories. Based on the complete set of reasoning trajectories, preference pairs are constructed for each sub-query question, thereby optimizing the reasoning path to enable the large language model to autonomously determine the best retrieval time. The process of dynamically selecting inference paths includes: For the subquery problem generated iteratively by the large language model, a confidence score of the large language model is generated, and then the corresponding reasoning action is selected and an honesty score of the large language model is generated, resulting in the corresponding reward function. During the reasoning process of exploring subquery questions iteratively based on Monte Carlo tree search, the score of each reasoning path is updated through a reward function. After the iteration is completed, reasoning paths with scores higher than a preset threshold are selected. The expression for the reward function is: ; in, Indicates the upper confidence interval; This represents the cumulative reward for the child nodes. This indicates the number of times the child node has been visited. Indicates the number of times the parent node has been visited; It is a constant; Represents the logarithmic function with base e; This represents the question-answering utility factor of a large language model. Indicates the use of historical states Assess the current issues The model confidence score, with a value range of ; Indicates the evaluation of intermediate answers The model's honesty score.

2. The method for determining a dynamic retrieval decision scheme based on Monte Carlo tree search as described in claim 1, characterized in that, The reasoning action can be any one of the following: direct answer, retrieval answer, query transformation, and final answer.

3. The method for determining a dynamic retrieval decision scheme based on Monte Carlo tree search as described in claim 2, characterized in that, If the confidence score of the large language model exceeds the preset confidence score threshold, the user is allowed to choose between a direct answer or a final answer action; otherwise, a retrieval answer or query transformation action is triggered.

4. The method for determining a dynamic retrieval decision scheme based on Monte Carlo tree search as described in claim 2, characterized in that, When the reasoning action is a direct answer or a final answer, the honesty score of the large language model ranges from [value missing]. When the reasoning action is to retrieve an answer or transform a query, the honesty score of the language model is 1.

5. The method for determining a dynamic retrieval decision scheme based on Monte Carlo tree search as described in claim 1, characterized in that, The expression for the reward function is obtained by summing the upper confidence interval and the question-answering utility factor of the large language model; the question-answering utility factor of the large language model is the product of the confidence score and honesty score of the large language model and a constant factor.

6. A dynamic retrieval decision scheme determination system based on Monte Carlo tree search, employing the dynamic retrieval decision scheme determination method based on Monte Carlo tree search as described in any one of claims 1-5, characterized in that, include: The reasoning process start point acquisition module is used to receive the initial query question and use it as the starting point of the reasoning process to trigger the invocation of the large language model operation; the reasoning process is pre-constructed as a Markov decision process. The reasoning path dynamic optimization module is used to call the large language model to perform subquery generation and action selection operations. It explores the reasoning actions of the subquery problem iteratively based on Monte Carlo tree search, and dynamically filters the reasoning path by considering the confidence score and honesty score of the large language model to obtain a complete set of reasoning trajectories. The optimal retrieval timing autonomous judgment module is used to construct preference pairs for each sub-query question based on the complete inference trajectory set, thereby optimizing the inference path and enabling the large language model to autonomously judge the optimal retrieval timing. The expression for the reward function is: ; in, Indicates the upper confidence interval; This represents the cumulative reward for the child nodes. This indicates the number of times the child node has been visited. Indicates the number of times the parent node has been visited; It is a constant; Represents the logarithmic function with base e; This represents the question-answering utility factor of a large language model. Indicates the use of historical states Assess the current issues The model confidence score, with a value range of ; Indicates the evaluation of intermediate answers The model's honesty score.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the dynamic retrieval decision scheme determination method based on Monte Carlo tree search as described in any one of claims 1-5.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps in the dynamic retrieval decision scheme determination method based on Monte Carlo tree search as described in any one of claims 1-5.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the dynamic retrieval decision scheme determination method based on Monte Carlo tree search as described in any one of claims 1-5.

Images