A method and system for truncated-overwritten large model multi-chain reasoning optimization

Abstract

The present application relates to the technical field of artificial intelligence, and more particularly to a large model multi-chain reasoning optimization method and system based on truncation-overwrite, aiming at the problem that current thinking chains lack effective monitoring and intervention, a multi-chain fusion reasoning method combining thinking chain score feedback, truncation-overwrite module and MCTS search strategy is proposed, the method comprises: obtaining an initial thinking chain of user input text; taking the initial thinking chain as a root node, calling a thinking chain evaluator to obtain the score of the root node and positioning the weak step; in each round of search: using the truncation-overwrite module to reconstruct a thinking chain according to the weak step of the node with the maximum confidence upper bound selected and taking it as an extended node; obtaining the score of the extended node and positioning the weak step; modifying the scores of all ancestor nodes of the extended node by back propagation of the score of the extended node; selecting the thinking chain of the node with the highest score as the answer output; a two-stage basic model fine-tuning method is also proposed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for optimizing large-scale multi-chain inference based on truncation-overwrite. Background Technology

[0002] In recent years, with the continuous improvement in the scale and capabilities of large language models, optimizing the multi-step inference process of these models to enhance their performance in complex tasks has become a recognized important research direction in academia and industry. Representative optimization path methods mainly include:

[0003] Method 1: Diagnosis and Correction of Intermediate Signals. This method diagnoses and corrects intermediate signals during the inference process. Methods such as SelfCheck can perform real-time self-evaluation, rewriting, and consistency checks at each step of the inference chain, thereby promptly identifying and correcting local logical errors. Furthermore, mechanisms such as information density and information gain are widely used to quantify the marginal contribution of each step to the final result, facilitating the identification and removal of redundant or inefficient steps and improving the overall quality and efficiency of the inference chain.

[0004] However, this method neglects the dynamic correction and reconstruction capabilities of the entire reasoning chain (i.e., the thought chain). Current intermediate signal diagnosis and correction methods, such as SelfCheck and Information Density, can locate and mark local errors in the reasoning chain, but most solutions provide relatively simple signals or lack interpretable evidence. Furthermore, these methods only support rewriting single-step content, lacking the ability to dynamically truncate, overwrite, and reconstruct the entire reasoning chain at any point, and rely too heavily on the model's own generation quality. For complex tasks or long-chain reasoning scenarios, single-point error correction is insufficient to completely improve the overall reasoning quality; local errors often lead to the collapse of the entire chain, affecting the reliability of the final answer.

[0005] Method 2: Inference Path Search and Multi-Chain Fusion. High-quality inference trajectory data is crucial for improving the multi-step inference capabilities of large models. Inference path search and multi-chain fusion methods have become core directions for structured inference optimization. Methods such as Self-Consistency (SC) use answer-level voting to achieve fusion and improve the robustness of answers. Building on this, the Minimum Consistent Revision (MCR) method directly enhances the model's capabilities by focusing on the Chain-of-Thought (CoT) path rather than the answer and performs efficient semantic fusion. In addition, policy search techniques such as Monte Carlo Tree Search (MCTS) can explore multiple inference branches in parallel and guide the model to generate logically consistent and efficient inference chains through trajectory scoring, path pruning, and process reward models.

[0006] However, this multi-chain fusion method suffers from redundancy and low information utilization. The Self-Consistency method relies heavily on coarse-grained strategies such as majority voting and trajectory-level merging, making it difficult to fully leverage the complementary advantages of high-quality segments across multiple inference chains, resulting in low effective information utilization and wasted redundant computational resources. Existing path search schemes such as Monte Carlo Tree Search (MCTS) often directly eliminate entire paths when encountering local errors, lacking efficient intra-chain repair and inter-chain fragment recombination mechanisms, making it difficult to achieve optimal utilization of the inference space. While methods like MCR that directly fuse CoT paths improve fusion efficiency, they also easily expose noise and errors in the inference process, potentially leading to misjudgments by the model.

[0007] Furthermore, the generation mechanisms based on policy search in related technologies suffer from problems such as contextual interference and insufficient information coherence. In traditional policy search processes such as Monte Carlo Tree Search (MCTS), the generation of each inference branch is often independent and discrete, making it susceptible to influence from external context or noise fragments, resulting in insufficient continuity of the inference chain, logical closure, and overall semantic consistency. From a probabilistic perspective, the high-order dependencies between branch contexts are not effectively modeled, further affecting the coherence and global quality of inference. Summary of the Invention

[0008] The present invention aims to at least solve the technical problems existing in the prior art, and provide a large model multi-chain inference optimization method and system based on truncation-overwrite.

[0009] In a first aspect, the present invention provides a large-scale multi-chain inference optimization method based on truncation-overwrite, the method comprising: acquiring user input text; calling a generation agent to initiate inference on the user input text to obtain an initial thought chain; using the initial thought chain as the root node of a Monte Carlo search tree; calling a thought chain evaluator to obtain the score of the root node and locate the weak steps of the root node; calling an MCTS searcher to perform multiple rounds of iterative search until the search termination condition is reached; in each round of iterative search, performing the following: calculating the upper confidence bound of each node on the current Monte Carlo search tree and selecting the node with the largest upper confidence bound; using a truncation-overwrite module to reconstruct a thought chain based on the weak steps of the selected node, and using the reconstructed thought chain as an extended node; calling the thought chain evaluator to obtain the score of the extended node and locate the weak steps of the extended node; modifying the scores of all ancestor nodes of the extended node on the Monte Carlo search tree by backpropagating the scores of the extended node; when the search termination condition is reached, selecting the thought chain of the node with the highest score on the Monte Carlo search tree as the answer and outputting the answer.

[0010] The beneficial effects of the above technical solution are as follows:

[0011] In the multi-round iterative search performed by the MCTS searcher, a thought chain evaluator was designed to obtain the node score and locate the weak steps of the node. This overcomes the lack of structured feedback on the reasoning path in related technologies, realizes the quality of accurate identification of thought chains and their steps in multi-chain fusion, and enhances the controllability and pertinence of the reasoning process.

[0012] In the multi-round iterative search performed by the MCTS searcher, the truncation-overwrite module is used to solve redundant noise. The thought chain is reconstructed based on the weak steps of the selected nodes. That is, high-quality intermediate steps are dynamically truncated and information is extracted to reshape the complete thought chain, avoiding noise interference and improving path continuity, thus making up for the limitations of existing strategy search methods in terms of information utilization.

[0013] In the multi-round iterative search performed by the MCTS searcher, the search is carried out in a structurally controllable space and the truncation-overwrite mechanism is invoked, which effectively realizes the reconstruction and expansion of the inference chain. Unlike the statically generated path of the traditional MCTS, this application strengthens the information sharing and local repair capabilities between paths. Finally, the scores of the extended nodes are modified by backpropagation of the scores of the extended nodes on all ancestor nodes in the Monte Carlo search tree.

[0014] To address the issues of contextual interference and insufficient information coherence in existing strategy-based generation mechanisms, this application differs from the common Monte Carlo search optimization process. Each node in the Monte Carlo search tree represents a complete thought process (i.e., a thought chain) rather than an intermediate execution step, thereby ensuring the coherence of the chain.

[0015] The above method can effectively monitor and optimize the thought process chain through structured positioning, integrate intermediate information and remove noise, so as to realize the cycle of large model from inference deployment to continuous optimization, and finally output a complete and high-quality answer to the user.

[0016] Secondly, this invention provides a large-scale multi-chain inference optimization system based on truncation-overwrite, the system comprising:

[0017] The model layer includes a base model, and based on the base model, generative agents, reflective generative agents, extraction agents, reflective agents, and thinking agents are obtained.

[0018] The inference optimization layer executes the steps of the large model multi-chain inference optimization method based on truncation-overwrite as described in the first aspect of the present invention.

[0019] The data layer stores the user input text, answers, and reflection information obtained after reasoning optimization by the reasoning optimization layer, forming a reasoning dataset;

[0020] The reasoning ability internalization layer uses the reasoning dataset to fine-tune the base model and then sends the fine-tuned base model to the model layer.

[0021] Preferably, the reasoning ability internalization layer uses a two-stage SFT fine-tuning method to fine-tune the basic model.

[0022] In addition to the beneficial technical effects of the steps in the large-model multi-chain inference optimization method based on truncation-overwrite described in the first aspect, the above system also has the following technical effects:

[0023] Based on user feedback, the corresponding reflection results and the optimized thought chain (i.e. the answer) are stored synchronously in the data layer. As the amount of data gradually accumulates, the reasoning and reflection capabilities can be further internalized into the model parameters through two stages of continuous capability injection, driven by the data layer, so as to achieve the self-evolution and closed-loop optimization of the model capabilities.

[0024] By employing supervised fine-tuning and LoRA efficient parameter fine-tuning techniques, combined with high-quality inference data generated by the aforementioned multi-chain fusion mechanism, a phased capability injection system is constructed to enhance the ability to cope with uncertainty and inference errors, realizing a closed-loop process from deployment, interaction to optimization iteration;

[0025] This invention adopts an optimization approach that moves from chain-like reasoning to reflective ability, further enhancing the model's structured reasoning capabilities and guiding the model to self-reflect and self-correct. Unlike traditional methods that only optimize the final answer, this method monitors and optimizes the thought chain through structured positioning, integrating intermediate information and removing noise to obtain high-quality answers. It accurately identifies and corrects vulnerabilities in intermediate nodes, significantly improving the model's performance in complex reasoning tasks. Furthermore, by internalizing external monitoring and intervention methods into the model parameters, the invention enables the model to spontaneously generate low-illusion responses. The invention also proposes a two-stage basic model fine-tuning method, employing an optimization approach that moves from chain-like reasoning to reflective ability to further enhance the model's structured reasoning capabilities and guide the model to self-reflect and self-correct. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a preferred embodiment of the large-model multi-chain inference optimization method based on truncation-overwrite in this invention.

[0027] Figure 2 This is a schematic diagram of the processing procedure of the truncation-overwrite module in a preferred embodiment of the present invention;

[0028] Figure 3 This is an architecture diagram of a large-model multi-chain inference optimization system based on truncation-overwrite in a preferred embodiment of the present invention. Detailed Implementation

[0029] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0030] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0031] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0032] This invention provides a large-model multi-chain inference optimization method based on truncation-overwrite, the execution subject of which includes, but is not limited to, at least one of electronic devices such as servers and terminals that can be configured to execute the method provided in the embodiments of this application. In other words, the large-model multi-chain inference optimization method based on truncation-overwrite can be executed by software or hardware installed on terminal devices or server devices. Servers include, but are not limited to, single servers, server clusters, cloud servers, or cloud server clusters. Servers can be independent servers or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0033] In a preferred embodiment, the flowchart of a large-model multi-chain inference optimization method based on truncation-overwrite provided by the present invention is as follows: Figure 1 As shown, it includes:

[0034] Step S1: Obtain the user input text, call the generation agent to start reasoning on the user input text to obtain the initial thought chain.

[0035] For example, the user input text is a natural language question or task description submitted by the user through an input device, which is not limited to a keyboard, touch screen, or voice recognition module.

[0036] Understandably, referring to Figure 3 The diagram shows the architecture of a large-scale multi-chain inference optimization system based on truncation-overwrite. The inference optimization layer receives user input text and calls the underlying generation agent to initiate inference on the user input text. Specifically, the inference optimization layer guides the generation agent's base model to generate an initial thought chain based on a preset thought chain prompt template. These prompts, through structured step-by-step inference requirements, explicitly guide the base model to systematically analyze and progressively solve the problem or task description in the user input text. Each task type (such as mathematical or general reasoning) has a specially customized thought chain prompt template, covering problem analysis, solution planning, detailed execution, and result verification. The thought chain prompt template emphasizes logical rigor, information completeness, and standardized terminology, and requires the output of inference steps and the final answer in JSON format to ensure the traceability and structured expression of the inference process.

[0037] In this embodiment, the base model is a non-reasoning large language model, which is not limited to the Qwen2.5 series model, DeepSeek-V3, Qwen-7B, or Qwen-14B. The base large language model, combined with prompt word templates and toolchains with different functions, can construct intelligent agents with different functions and roles, that is, agents with different functional roles, such as the generative agent, reflective generative agent, extraction agent, reflective agent, and thinking agent in this application.

[0038] Preferably, in addition to recording the multi-step reasoning steps and intermediate conclusions for generating the proxy output, the system also simultaneously records the token probability, local confidence level, and model self-score for each step. This fine-grained information provides foundational data for the subsequent thought chain evaluator to assess the quality of the initial thought chain and optimize the path.

[0039] Step S2: Using the initial thought chain as the root node of the Monte Carlo search tree, call the thought chain evaluator to obtain the score of the root node and locate the weak steps of the root node.

[0040] The Monte Carlo search tree is the search tree in the Monte Carlo Tree Search (MCTS) algorithm. (See also...) Figure 3Nodes n1, n2, n3, n4, n5, and n6 form a Monte Carlo search tree, including a root node n1 and five non-root nodes. Each node is connected by an edge. Each node stores information including the number of visits and the score of the corresponding thought chain, i.e., the node's score. The initial value of the node's score is obtained through the thought chain evaluator, but it may be modified in subsequent MCTS search based on the search results.

[0041] For example, please see Figure 2 The upper left corner displays a thought chain (also called a reasoning chain), which shows the complete reasoning steps for answering questions or tasks input by the user, including five steps: Step 1-Step 5. The thought chain evaluator assesses the score of each step in the thought chain, and then combines the scores of all steps to obtain the overall score of the thought chain. The step with the lowest score is identified as a weak step.

[0042] Step S3: Call the MCTS searcher to perform multiple rounds of iterative search until the search termination condition is met. In each round of iterative search, the following is executed:

[0043] Step S31: Calculate the upper confidence bound of each node in the current Monte Carlo search tree, and select the node with the largest upper confidence bound;

[0044] Step S32: Use the truncation-overwrite module to reconstruct a thought chain based on the weak steps of the selected nodes, and use the reconstructed thought chain as an extended node.

[0045] Step S33: Call the MindChain evaluator to obtain the score of the extended node and locate the weak steps of the extended node;

[0046] Step S34: Modify the scores of all ancestor nodes of the expanded node in the Monte Carlo search tree by backpropagating the scores of the expanded node. The ancestor nodes of a node include the node's parent node, the parent node's parent node, and so on up to the root node.

[0047] In this embodiment, the search termination condition is not limited to the number of iterations reaching a preset maximum number of iterations, the search time exceeding a preset search time threshold, or the failure to find a better reward mind chain after multiple rounds of searching.

[0048] Step S4: When the search termination condition is met, select the thought chain of the node with the highest score on the Monte Carlo search tree as the answer and output the answer.

[0049] In this embodiment, to obtain accurate answers more efficiently, preferably, after performing one round of iterative search, for each node's candidate child nodes (i.e., two or more child nodes (not parent nodes) directly connected to that node), such as... Figure 3(where n4 and n5 are both candidate child nodes of n3) Perform adaptive pruning based on reward density, as shown in the following formula:

[0050]

[0051] This indicates that a pruning operation is performed on the candidate child node set of node n. The above formula represents the pruning operation performed on the candidate child node set of node n. In, retain the satisfaction The candidate child node c. This represents the score of candidate child node c. This indicates the number of times candidate child node c has been visited. This indicates retrieving the candidate child node set. All candidate child nodes The median of the values, This represents the proportionality coefficient. Indicates candidate child nodes The rating, Indicates candidate child nodes The number of visits.

[0052] In a preferred embodiment, for newly generated thought chain nodes (i.e., the root node corresponding to the initial thought chain and the extended node corresponding to the reconstructed thought chain), the system calls the thought chain evaluator to conduct a detailed analysis of each step of the complete thought chain of the node to obtain a score.

[0053] The process by which the MindChain evaluator obtains node scores includes:

[0054] Step A1: Calculate the language model confidence of each step in the node's thought chain based on the log probabilities (logprobs) when the node's thought chain generation agent generates the node's thought chain.

[0055] In this embodiment, for the root node, the generation proxy for the node's thought chain is the generation proxy in step S1; for the extended node, the generation proxy for the node's thought chain is the thought proxy, please see... Figure 3 The truncation-overwrite mechanism section. The log probabilities (logprobs) are used by the thought chain generation agent of each node to calculate a probability value for each generated token (word or sub-word), representing the likelihood of that token in the current context.

[0056] In this embodiment, preferably, each step in the thought chain of a node includes one or more lexical units, and the steps in the thought chain of a node... Language model confidence The calculation formula is:

[0057]

[0058] in, Steps in a thought chain representing nodes The set consisting of all lexical units; The proxy generator generates tokens representing the thought chain of nodes. The logarithmic probability; Steps in a thought chain representing nodes The length of this step is... The total number of tokens, which is the number of steps calculated using the same word segmentation method as the logprob calculation. The number of lexical units; Indicates word elements The weights;

[0059]

[0060] in, This represents the number of steps in the thought chain of a node; This indicates that the Monte Carlo search tree includes terms. The number of thought chains; Indicates word elements Steps in the thought chain at the node The number of times it appears in; Steps in a thought chain representing nodes The number of lexical units included and They have the same meaning; they are both steps. The total number of tokens is simply used for one position in length normalization and the other in word frequency normalization; they are essentially the same.

[0061] The confidence level of the above language model The calculation formula incorporates inverse document frequency and term frequency normalization terms to suppress scoring interference from conjunctions and long contexts, emphasizing information density. The formula also includes length normalization to prevent long sentences from gaining an unfair advantage.

[0062] Step A2: Calculate the information flow score for each step in the node's thought chain. The information flow score is used to measure the smoothness of the local semantics of each step in the node's thought chain. The specific method is to calculate semantic similarity.

[0063] Preferably, the steps in the thought chain of a node Information flow scoring is based on steps First information flow scoring and steps The second information flow score was obtained;

[0064] step The first information flow rating is:

[0065]

[0066] step The second information flow score is:

[0067]

[0068] in, Indicate steps The set of adjacent steps; express intermediate steps The number of adjacent steps; Indicate steps Word vectors; Indicate steps The Word vectors from adjacent steps; express The L2 norm; express The L2 norm; Indicate the calculation steps word vectors Word vectors of user input text The cosine similarity. Word vectors are generated by word embedding models and are used to measure the information relevance at different steps in the semantic space.

[0069] In this embodiment, the steps Information flow scoring is a step The weighted sum of the first and second information flow scores.

[0070] Step A3: Based on the reflection generation agent, obtain the reflection score of each step in the thought chain of the node, and at the same time calculate the structural position score of each step in the thought chain of the node. Use the structural position score of each step to correct the reflection score of each step to obtain the metacognitive score of that step.

[0071] In this embodiment, the reflection generation agent is generated by combining a basic model with reflection prompt templates and a toolchain. The input to the reflection generation agent consists of reflection prompts generated based on the reflection prompt templates, scoring samples, and scoring rules. Thus, guided by the reflection prompts, the reflection generation agent evaluates the logical consistency, relevance, completeness, depth, and structural rationality of the reasoning steps in the thought chain of nodes, obtaining a reflection score for each step. More preferably, to suppress abnormally high scores and overly even distributions, reflection scores are assigned certain weights, and the weighted reflection score is obtained by multiplying the reflection score by the weighted reflection score. Reflection Score Weights The calculation formula is:

[0072]

[0073] This represents the system's preset reflection weights; Represents the reflection fractional variance of all steps in the thought chain of a node; Indicates the variance threshold; This represents the average reflection score across all steps of the thought chain at a node. This represents the threshold for the average reflection score. The formula above indicates that when the variance of the reflection scores for all steps in the thought chain of a node is less than [a certain value], [the threshold is reached]. Furthermore, the average reflection score of all steps in the thought chain of a node is greater than [a certain value]. At the same time, the weight of reflection scores is further reduced to curb abnormally high overestimation scores and overly even distributions.

[0074] In this embodiment, in the structural position score of each step in the thought chain of the computing node, for step s, its structural position score is:

[0075]

[0076] in, This indicates the step number of step s in the node's thought chain. This represents the number of steps in the thought chain of a node.

[0077] For example, in step A3, the reflection score of each step is corrected using the structural position score of each step to obtain the metacognitive score of that step. This includes: weighted fusion of the reflection score and structural position score of each step to obtain the metacognitive score of that step, as shown in the formula above. The weight of the reflection score is... or The weight of the structural location score is then... or .

[0078] Step A4: Obtain the score for each step based on the language model confidence, information flow score, and metacognitive score.

[0079] Step A5: Calculate the sum of the scores of all steps in the thought chain of the node, and use the sum as the score of the node.

[0080] In this embodiment, more preferably, the thought chain evaluator identifies the step with the lowest score in a node's thought chain as the node's weak step.

[0081] In a preferred embodiment, in step S31, the step of calculating the confidence upper bound for each node in the current Monte Carlo search tree, the node... upper confidence level for:

[0082]

[0083] in, Represents a node Current rating; Represents a node Number of visits; Represents a node parent node Number of visits; It is the first weight; Indicates the stability coefficient; Indicates the second weight; Indicates the third weight; Indicates the fourth weight; Indicates the fifth weight; Represents a node in the Monte Carlo search tree The depth, i.e., the number of edges traversed; it is the same concept as "path length", which can be directly understood as the level of the node in the tree; Represents a node The number of times the rating was modified through backpropagation; Represents a node in the Monte Carlo search tree The length of the corresponding thought chain is defined as the number of edges from the root node to the current node; Represents a node Is it a newly generated extended node from the truncation-overwrite module? If the node... If it is a newly generated extended node by the truncation-overwrite module, then... =1, if node If it is not a newly generated extended node by the truncation-overwrite module, then =0.

[0084] In this embodiment, the node parent node Represents nodes Directly connected ancestor nodes. The above upper bound of confidence. In the calculation formula, the number of visits to the parent node is used instead of the total number of visits. This method uses the number of visits to the parent node instead of the global count, emphasizing the pressure of local exploration and enhancing the deep structure optimization capability. This balances exploration and utilization and improves the interpretability of path selection.

[0085] In a preferred embodiment, in step S34 above, during the step of modifying the scores of all ancestor nodes of the extended node in the Monte Carlo search tree through backpropagation, the extended node... Ancestor node The revised scoring formula is as follows:

[0086]

[0087] in, Represents ancestor nodes Revised rating; Represents ancestor nodes The original score; Indicates an extended node After generation, the score obtained by calling the MindChain evaluator is called; Indicates the depth reward amplification factor; Represents ancestor nodes Depth of the Monte Carlo search tree; Indicates an extended node Ancestor Nodes The distance between them; Indicates the propagation attenuation factor of Power of 1.

[0088] In this implementation, during each round of search iteration expansion, after the thought chain evaluator evaluates the expanded node, it obtains the score of each step in the node's thought chain and the overall node score R(C), also known as the comprehensive score, and records the corresponding content for backpropagation. The score is propagated back to its parent node, the parent node's parent node, ..., the root node. The aforementioned extended nodes... Ancestor node The modified scoring formula can enhance the reward effect of deep reasoning paths, guiding the MCTS searcher to delve deeper into high-value reasoning links.

[0089] In a preferred embodiment, please refer to Figure 2 In step S32, a thought chain is reconstructed based on the weak steps of the selected nodes using the truncation-overwrite module, including:

[0090] Step S321: Use the extraction agent to extract the steps before the weak step from the thought chain of the selected node and record them as the preceding steps; this is equivalent to cutting off the chain at the weak step.

[0091] Step S322: Use the reflection agent to obtain reflection information on weak steps;

[0092] Step S323: Input the positions of the preceding steps, reflection information, and weak steps into the thinking agent, and the thinking agent reconstructs a thinking chain.

[0093] In this embodiment, the extraction agent is obtained by combining the basic model with extraction prompt word templates and toolchains. Its inputs are extraction prompt words and the location information of the weak step, and its output is the steps in the thought chain of the selected node before the weak step.

[0094] In this implementation, the reflective agent is obtained by combining a basic model with reflective prompt templates and toolchains. Its input is the thought chain and weak steps of the nodes, and its output is reflective information.

[0095] In this implementation, the thinking agent obtains the reconstructed thinking chain by combining a basic model with a reconstruction prompt template and a toolchain. Its inputs include the positions of prerequisite steps, reflection information, and weak points, and its output is the reconstructed thinking chain. During execution, the thinking agent aims to maximize the score of extended nodes, performing chain segment reconstruction to obtain the reconstructed thinking chain. :

[0096]

[0097] In the solution space Maximize the interior; This indicates the intention to obtain the expected value; Indicates based on the current thought chain Weak steps Reconstruction is carried out to obtain the reconstructed thought chain. The highest score is obtained through the Mind Chain Evaluator.

[0098] For example, please see Figure 3 The thought chain has k+1 steps, where the thought chain evaluator identifies the k-th step as the weakest step. In step S321, the chain phase is performed, retaining only the first k-1 steps.

[0099] The process described in this implementation iterates continuously within MCTS, ensuring the continuity of the logical chain and the validity of the information. Unlike DeepSeek-Prover-V1.5, which primarily relies on theorem prover feedback and continuation mechanisms, the truncation-overwriting module in this implementation focuses on natural language processing tasks, emphasizing the ability to truncate and reconstruct the original thought chain. This allows the reasoning process to retain structural continuity while possessing stronger generalization and adaptability. It is worth noting that the emphasized truncation only occurs in the information extraction step, while overwriting reshapes a complete new thought chain based on the extracted information, rather than continuing the original thought chain. The resulting new thought chain and its corresponding information are then integrated as a complete new node.

[0100] This invention also discloses a large-model multi-chain inference optimization system based on truncation-overwrite, in a preferred embodiment, such as... Figure 3 As shown, the system includes:

[0101] The model layer includes a base model, which is used to obtain generative agents, reflective generative agents, extraction agents, reflective agents, and thinking agents.

[0102] The inference optimization layer executes the steps of the above-described large-model multi-chain inference optimization method based on truncation-overwrite.

[0103] The data layer stores the user input text, answers, and reflection information obtained after reasoning optimization by the reasoning optimization layer, forming a reasoning dataset;

[0104] The reasoning ability internalization layer uses the reasoning dataset to fine-tune the base model and then sends the fine-tuned base model to the model layer.

[0105] Preferably, the reasoning ability internalization layer uses a two-stage SFT fine-tuning method to fine-tune the basic model.

[0106] In this implementation, the data layer plays a crucial pivotal role in data accumulation, sample organization, and capability transfer. The data layer not only continuously collects high-quality data generated during the inference optimization process but also provides solid support for the internalization and improvement of subsequent basic model capabilities, achieving a self-evolution and optimization loop for the system.

[0107] The data layer and the reasoning ability internalization layer work together to perform the following steps:

[0108] Step 201: Data accumulation and sample screening.

[0109] In each user interaction and reasoning optimization loop, the system automatically sets the triple (user input text Q, optimized optimal thinking, i.e., the answer) to the optimal solution. The reflection information (F) is synchronously written to the data layer to form a reasoning dataset. To ensure the quality of the capability-injected data, the system periodically selects reasoning data samples with an overall thinking chain score higher than the thinking chain score threshold, positive or remediable negative user feedback, and diversity and representativeness, ultimately constructing a high-quality reasoning dataset. And further subdivided into subsets of reasoning trajectories. and Reflection-Repair Subset .

[0110] The following describes how the reasoning ability internalization layer uses a two-stage SFT fine-tuning method to fine-tune the base model. SFT fine-tuning usually refers to Supervised Fine-Tuning, which is to adapt and optimize a pre-trained model (such as an LLM large language model) for downstream tasks through supervised learning. It is not limited to the two-stage LoRA parameter injection method, but includes steps 202 and 203.

[0111] Step 202: Internalization of reasoning ability and significant injection of LoRA parameters, i.e., the first stage.

[0112] The model capability injection process first begins with The training set is further mixed with English text and English mathematical content used during model training at a ratio of 20%. Then, the LoRA (Low-Rank Adaptation) parameter efficient injection mechanism is used to adjust the basic model parameters. Perform large weight matrix , The injection adjustment. Specifically, the training objective is to achieve the following mapping by minimizing the token-level cross-entropy loss:

[0113]

[0114] Where ΔLoRA(A1,B1) represents the subspace-based... , The low-rank parameter bias can significantly improve the model's ability in multi-step inference decomposition and structured output, producing the first-stage basic model version. .

[0115] Step 203: Injecting reflection and self-correction capabilities, i.e., the second stage.

[0116] Building upon the internalization of capabilities in the first phase, with As a sample, and The data is mixed at a 1:4 ratio, and then further superimposed with 20% of the training data to perform LoRA parameter matrix adjustments on the first-stage base model. , Injection, obtain parameter set The training objective of this stage is to cultivate the model's ability to self-reflect and correct itself, and on this basis, to deeply inject the model's self-correction capability.

[0117] Step 204: Closed-loop deployment and online self-evolution.

[0118] After completing the two-stage LoRA parameter injection, the final model The data is injected back into the model layer, completely replacing the old agent. The system continuously collects user interaction and feedback data, cyclically entering the data filtering and parameter re-injection stage, forming a self-evolving closed loop of "inference optimization - data collection - parameter injection - inference optimization", continuously improving the performance and adaptability of large models in complex multi-step inference scenarios.

[0119] The present invention can achieve the following technical effects:

[0120] 1. A More Efficient and Interpretable Optimization and Evaluation System: This invention utilizes a multi-dimensional thought chain evaluator based on interpretable signals and MCTS multi-agent collaborative search to not only achieve global-local consistency optimization of the inference path but also provide structured, hierarchical, high-quality training data for subsequent capability injection. Each step in the inference chain (i.e., the thought chain) can be tracked, analyzed, and located, allowing users and developers to intuitively understand the basis for model decisions and the optimization process. Simultaneously, through adaptive reward density pruning and an improved UCB structured scoring function, the data utilization rate and overall system efficiency of the inference process are significantly improved, ensuring that optimization resources are focused on high-value links (thought chains), thereby enhancing the reliability and generalization ability of the final inference results.

[0121] 2. Enhanced Reasoning Controllability and Adaptability: This invention proposes a reasoning optimization system based on multi-chain fusion and truncation-overwrite mechanisms for advanced application scenarios such as complex tasks and multi-step reasoning. The system can automatically generate detailed reasoning processes for the same user input question, achieving efficient integration and reasoning optimization of multi-path information through fine-grained evaluation, information extraction, and recombination. It is particularly suitable for scenarios requiring high interpretability, result consistency, and reasoning robustness, significantly improving the feasibility and transparency of the model in practical complex applications, while greatly enhancing the model's accuracy in complex logical problems.

[0122] 3. Multi-chain fusion-driven truncation-overwrite reasoning optimization mechanism: This invention uses multi-chain fusion as its core, and through fine-grained quality assessment of the thought chain, it achieves automatic location and targeted truncation, information extraction, and overwriting reconstruction of weak links in the chain. This mechanism not only overcomes the limitations of traditional single-chain reasoning, such as the inability to repair local errors and poor robustness of the reasoning chain, but also greatly improves information utilization efficiency and the consistency of reasoning results, providing a solid technical foundation for high-order intelligent reasoning for complex problems.

[0123] 4. A Global Path Optimization Method Based on MCTS and Multi-Agent Collaboration: The invention deeply integrates Monte Carlo Tree Search (MCTS) with a multi-agent collaborative mechanism to achieve global-local integrated dynamic optimization of the inference chain. The system utilizes mechanisms such as interpretability rewards, multi-dimensional pruning, and deep rewards to collaboratively drive iterative optimization of the inference chain within the solution space through the joint evaluation agent, reflective agent, information extraction agent, and rewriting agent. Furthermore, this invention provides customized improvements to the node backpropagation method and the UCB formula. This method significantly improves inference diversity, path quality, and search efficiency, ensuring system stability and optimal solution reachability under complex inference tasks.

[0124] 5. Integration of Multidimensional Interpretable Signals and Link Intervention Mechanism: This invention integrates multidimensional interpretable signals throughout the entire process of inference chain generation, evaluation, and optimization. These signals include language model confidence, information flow score, and metacognitive reflection score. Through quantitative and traceable methods, it accurately identifies potential risks and information breakpoints in the inference chain, using these as direct intervention criteria for operations such as pruning, truncation, and rewriting. This mechanism significantly enhances the interpretability and transparency of the inference process, effectively ensuring the quality of the inference chain and the controllability of the system output.

[0125] 6. A More Rational Capability Transfer and Model Self-Evolution Process: Unlike traditional single-stage fine-tuning, this invention introduces a two-stage capability injection system for the first time, specifically a two-stage parameter fine-tuning system that decouples reasoning ability and reflective correction ability. The first stage injects a large amount of LoRA weight matrix, focusing on the internalization of multi-step structured capabilities in the reasoning trajectory; the second stage further integrates the reflective correction link, significantly enhancing the model's self-correction and dynamic adjustment capabilities. The closed-loop design of the entire process ensures that the model's capabilities can continuously evolve to adapt to constantly changing practical needs. This layered capability injection scheme effectively solves the problem that single-objective fine-tuning cannot simultaneously address complex reasoning development and structural introspection, greatly improving the model's generalization ability and evolutionary space in complex dynamic scenarios.

[0126] 7. This invention can improve and optimize existing model responses, outputting higher-quality inference results and enhancing system reliability. Simultaneously, the method provides a reliable approach for collecting user feedback and accumulating high-quality data to automatically improve model performance, and it is compatible with mainstream reinforcement learning frameworks.

[0127] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0128] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for optimizing multi-chain inference in large models based on truncation-overwrite, characterized in that, The method includes: Obtain the user input text, call the generation agent to start reasoning on the user input text to obtain the initial thought chain; Using the initial thought chain as the root node of the Monte Carlo search tree, the thought chain evaluator is called to obtain the score of the root node and locate the weak steps of the root node. The MCTS searcher is invoked to perform multiple rounds of iterative search until the search termination condition is met. In each round of iterative search, the following is executed: Calculate the upper confidence bound for each node in the current Monte Carlo search tree, and select the node with the largest upper confidence bound; The truncation-overwrite module is used to reconstruct a thought chain based on the weak steps of the selected nodes, and the reconstructed thought chain is used as an extended node. Call the MindChain evaluator to obtain the score of the expansion node and locate the weak steps of the expansion node; Modify the scores of all ancestor nodes of the extended node on the Monte Carlo search tree by backpropagating the scores of the extended node. When the search termination condition is met, select the thought chain of the node with the highest score on the Monte Carlo search tree as the answer and output the answer. The process by which the MindChain evaluator obtains node scores includes: Calculate the language model confidence of each step in the node's thought chain based on the log probability of the proxy generating the node's thought chain. The information flow score for each step in the thought chain of a computing node is used to measure the smoothness of the local semantics of each step in the thought chain of the node. The reflection score of each step in the thought chain of a node is obtained by the reflection-generating agent. At the same time, the structural position score of each step in the thought chain of a node is calculated. The reflection score of each step is corrected by the structural position score of each step to obtain the metacognitive score of that step. The score for each step is obtained based on the language model confidence, information flow score, and metacognitive score for each step; The sum of the scores of all steps in the thought chain of a computation node is used as the node's score.

2. The method as described in claim 1, characterized in that, Each step in a node's thought chain includes more than one lexical unit. The steps in a node's thought chain... Language model confidence The calculation formula is: in, Steps in a thought chain representing nodes The set consisting of all lexical units; The proxy generator generates tokens representing the thought chain of nodes. The logarithmic probability; Steps in a thought chain representing nodes Length; Indicates word elements The weights; The in, This represents the number of steps in the thought chain of a node; This indicates that the Monte Carlo search tree includes terms. The number of thought chains; Indicates word elements Steps in the thought chain at the node The number of times it appears in; Steps in a thought chain representing nodes The number of lexical units included.

3. The method as described in claim 1, characterized in that, Steps in the thought chain of a node Information flow scoring is based on steps First information flow scoring and steps The second information flow score was obtained; step The first information flow rating is: step The second information flow score is: 。 4. The method as described in claim 1, characterized in that, The thought chain evaluator identifies the step with the lowest score in a node's thought chain as the node's weak step.

5. The method as described in claim 1, characterized in that, node upper confidence level for: in, Represents a node Current rating; Represents a node Number of visits; Represents a node parent node Number of visits; It is the first weight; Indicates the stability coefficient; Indicates the second weight; Indicates the third weight; Indicates the fourth weight; Indicates the fifth weight; Represents a node in the Monte Carlo search tree The depth; Represents a node The number of times the score was modified through backpropagation; Represents a node in the Monte Carlo search tree Corresponding to the length of the thought chain; Represents a node Is it a newly generated extended node from the truncation-overwrite module? If the node... If it is a newly generated extended node by the truncation-overwrite module, then... =1, if node If it is not a newly generated extended node by the truncation-overwrite module, then =0.

6. The method as described in claim 1, characterized in that, In the step of modifying the scores of all ancestor nodes of the expanded node in the Monte Carlo search tree through backpropagation, the expanded node... Ancestor node The revised scoring formula is as follows: in, Represents ancestor nodes Revised rating; Represents ancestor nodes The original score; Indicates an extended node After generation, the score obtained by calling the MindChain evaluator is called; Indicates the depth reward amplification factor; Represents ancestor nodes Depth of the Monte Carlo search tree; Indicates an extended node Ancestor Nodes The distance between them; Indicates the propagation attenuation factor of Power of 1.

7. The method as described in claim 1, characterized in that, The method of reconstructing a thought chain using the truncation-overwrite module based on the weak steps of selected nodes includes: Using an extraction agent, extract the steps preceding the weak steps from the thought chain of the selected nodes and record them as the preceding steps; Use reflective agents to obtain reflective information about weak steps; By inputting the locations of the preceding steps, reflection information, and weak points into the thinking agent, the thinking agent reconstructs a thinking chain.

8. A large-scale multi-chain inference optimization system based on truncation-overwrite, characterized in that, The system includes: The model layer includes a base model, and based on the base model, generative agents, reflective generative agents, extraction agents, reflective agents, and thinking agents are obtained. The inference optimization layer executes the steps of the large model multi-chain inference optimization method based on truncation-overwrite as described in any one of claims 1-7; The data layer stores the user input text, answers, and reflection information obtained after reasoning optimization by the reasoning optimization layer, forming a reasoning dataset; The reasoning ability internalization layer uses the reasoning dataset to fine-tune the base model and then sends the fine-tuned base model to the model layer.

9. The system as described in claim 8, characterized in that, The reasoning ability internalization layer uses a two-stage SFT fine-tuning method to fine-tune the basic model.

Images