Thinking chain reasoning early stop control method and system based on entropy-probe double signals

Abstract

The invention discloses a thinking chain reasoning early stop control method and system based on entropy-probe double signals, and the method comprises the steps: extracting the internal hidden state and external output probability distribution of a model in real time with a logic step as a unit in the process of generating a thinking chain through autoregression of a large-scale language model; utilizing a lightweight probe to map the hidden state into a reasoning convergence confidence coefficient, and calculating a step entropy as an uncertainty index; carrying out joint evaluation on the double-characteristic signals; carrying out truncation reasoning immediately through a fast channel when the signals are extremely strong, and carrying out multi-step consistency verification through a stable window mechanism when the signals tend to converge but fluctuate; and once an early stop condition is triggered, immediately terminating generation of a subsequent redundant thinking chain and outputting a final answer. According to the method, redundant steps can be effectively recognized and cut off, reasoning delay and calculation cost are remarkably reduced on the premise that base model parameters are not changed, the system robustness is improved while the accuracy is kept, and the error risk caused by excessive thinking is relieved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, specifically to the field of large model inference acceleration and efficiency optimization, and particularly to a method and system for controlling early stopping of thought chain inference based on entropy-probe dual signals. Background Technology

[0002] To alleviate the efficiency issues caused by long thought chains, existing technologies have proposed various training-free early stopping methods that dynamically intervene during the inference phase. One type of method is based on text-level heuristic rules, monitoring specific language markers during generation, such as hesitation or transition words like "Wait" and "Let me check," or explicitly inserting multi-turn intermediate answer verification prompts to detect whether the intermediate answers generated multiple times within a sliding window are consistent. When "multi-turn consistency" is detected or a large number of hesitation markers appear, it is determined that the model has entered a redundant stage or needs to be replanned, thus terminating subsequent inference early. This type of method is simple to implement and does not require changes to model parameters, but it is highly dependent on the form and style of the output text and the design of the prompt template. When the task type, language style, or prompt strategy changes, its stability and generalization ability are often insufficient, and frequent insertion of verification prompts also adds additional generation overhead.

[0003] Another type of approach utilizes the uncertainty signal in the model's output distribution to design early termination rules. A typical approach is to calculate Shannon entropy or semantic entropy at the end of each inference step based on the probability distribution of the next word, and use the average entropy across steps as a quantification of the model's current "confusion level." When the entropy value for multiple consecutive steps remains below a certain threshold, it is considered that the model is highly confident in the current inference path, and early termination can be triggered to save on the generation of subsequent redundant tokens. This type of method does not depend on specific text content, operates only on logits, has low additional computational cost, and can be easily embedded into existing decoding processes. In many scenarios, it can effectively avoid premature termination during obviously high-entropy, highly uncertain stages.

[0004] Recent work has attempted to extract more direct "correctness" and "convergence" signals from the hidden states of models, such as training probe models based on hidden layer representations for self-verification or confidence assessment. These methods demonstrate, to some extent, that the internal states do indeed contain separable information about the quality of the inference path. However, existing probe techniques are mostly used for offline analysis or posterior answer discrimination, often making a one-time judgment on the entire input-output pair, lacking a mechanism for online decision-making at the step-by-step thought chain granularity; some methods require retraining or fine-tuning large discriminant networks, resulting in high engineering integration costs; and some works focus only on a single signal, such as pure entropy or pure probe scores, and have not yet formed a mature multi-signal joint decision-making framework.

[0005] In summary, although existing technologies have alleviated the efficiency problem of long thought chain reasoning to some extent, they still have the following shortcomings: (1) Many methods rely on surface text patterns or heuristic rules, are sensitive to task form and language style, and have limited robustness; (2) Entropy-based uncertainty measures can only reflect the confidence level of the model and cannot reliably distinguish between "confident and correct" and "confident but wrong" situations, and are prone to premature exit on the wrong path; (3) Probe-type methods that utilize hidden states mostly stay at the level of offline analysis or coarse-grained discrimination, and lack an engineering scheme to gradually and stably control the length of the thought chain during the reasoning process; (4) Existing early stopping methods are mostly designed around a single signal and have not fully explored the complementarity between external probability distribution signals and internal representation signals, making it difficult to achieve a more robust trade-off between reasoning accuracy and reasoning overhead. Summary of the Invention

[0006] The purpose of this invention is to address the problems in existing technologies where early stopping strategies rely on surface text or a single entropy signal, have difficulty distinguishing between "confident and correct" and "confident but wrong," and suffer from excessive thinking leading to lengthy reasoning and unstable conclusions. This invention provides a method for controlling early stopping in thought chain reasoning based on entropy-probe dual signals. Without modifying large model parameters, it achieves fine-grained dynamic control over the thought chain generation process, significantly reducing reasoning token consumption and latency while maintaining or even improving reasoning accuracy, thereby enhancing the reliability and resource utilization efficiency of the reasoning process.

[0007] One aspect of the present invention provides a method for controlling early termination of thought chain reasoning based on entropy-probe dual signals, comprising:

[0008] Step S1: Obtain the set of reasoning tasks to be processed and the pre-trained large language model, set the thought chain generation format and logic step boundary recognition rules, so that the model outputs the thought chain text step by step in the reasoning process according to the numbering format (1), (2), (3), etc., and can detect the boundary of the current logic step at the end of each numbered step;

[0009] Step S2: During the training phase, the task set is input into the pre-trained language model to generate a complete thought chain reasoning process, record the output probability distribution and intermediate hidden states corresponding to each logical step, and automatically label the path correctness information and reasoning completion status information of each logical step based on the thought chain generation format and logical step boundary recognition rules to construct a step-level training sample set.

[0010] Step S3: Extract hidden state signals for each logic step to obtain step-level hidden states; use whether the current state is a reasoning completion state as a label to train the probe, and output the confidence score of whether the current logic step is sufficient to support the correct answer based on the step-level hidden states;

[0011] Online reasoning stage

[0012] Step S4: Input the target problem into the pre-trained language model and generate logical steps step by step under the set thought chain prompts; at the end of each logical step, calculate the average entropy of the step within that step as an external uncertainty signal, and extract the corresponding step-level hidden state and input it into the pre-trained probe model to obtain the probe confidence of the current logical step as an internal correctness signal.

[0013] Step S5: Set a fast early stop strategy based on the step average entropy and probe confidence: When the step average entropy of the current logical step is lower than the preset very low entropy threshold and / or the probe confidence is higher than the preset very high confidence threshold, it is determined that the model is in a very high confidence state, and early stop control is triggered directly in this logical step to terminate the generation of subsequent thought chains, and guide the model to output the final answer based on the current context.

[0014] Step S6: When the fast early stopping condition is not met, a joint judgment is made on several consecutive logical steps according to the stable early stopping strategy: Maintain a sliding window of fixed length, collect the step average entropy and probe confidence of each logical step in the window, when the probe confidence of all logical steps in the window is not lower than the preset stability threshold and the step average entropy is generally at a medium-low level, the current logical step is regarded as a candidate early stopping point, and a lightweight consistency check is performed to confirm whether the reasoning result is stable. When the check passes, the stable early stopping control is triggered and the final answer is output. When the check fails, the early stopping is abandoned and subsequent logical steps are generated.

[0015] Step S7: During the reasoning process, continuously monitor the number of generated thought chain tokens and the preset token budget or maximum thought chain length. When the budget limit or maximum length is reached but early stop control is not triggered, generate a complete thought chain by default and output the final answer, so that the entire reasoning process is safely terminated under the given resource constraints.

[0016] Furthermore, the external uncertainty mentioned in step S4 is based on the token-level entropy, which measures the probability distribution uncertainty of the large language reasoning model at the t-th generation position using Shannon entropy H_t, defined as follows:

[0017] Formula (1)

[0018] (1)

[0019] in:

[0020] t represents the index of the t-th token currently generated;

[0021] V represents the vocabulary set output by the model;

[0022] v represents any candidate token in the vocabulary V;

[0023] This represents the conditional probability that "the next token will have a value of v" at position t. = ( = v | , q);

[0024] log represents the logarithmic function, and the logarithm with the natural logarithm as the base is usually chosen;

[0025] Let be the Shannon entropy at position t. A larger value indicates a more dispersed probability distribution at that position and a more hesitant model. The smaller the value, the more concentrated the distribution and the more confident the model.

[0026] Furthermore, the external uncertainty signal introduces the step average entropy. , used to describe the overall entropy level of the i-th logical step;

[0027] Formula (2):

[0028] (2)

[0029] in:

[0030] This represents the i-th logical step, such as a whole paragraph of reasoning text in the format “(1)…(2)…(3)…”;

[0031] T( () indicates steps The set of token location indices contained therein;

[0032] |T( )| represents set T( The size of ) is the step The number of tokens in the middle;

[0033] Let be the Shannon entropy at position t, with the same meaning as in formula (1);

[0034] For steps The average entropy, equal to the arithmetic mean of the entropy values ​​of all tokens within that step, is used as an external uncertainty signal for that step.

[0035] The step-level hidden state described in step S4 is obtained by averaging the hidden vectors in step S4. ,

[0036] Formula (3):

[0037] (3)

[0038] in:

[0039] ∈ The last hidden vector representation of the large language reasoning model at position t has a dimension of d.

[0040] T( ),|T( The meaning of )| is the same as in formula (2);

[0041] Indicate steps The step-level hidden state is the hidden vector of all tokens within the step. The vector obtained by taking the arithmetic mean can be understood as the overall internal representation of the model after completing this step of reasoning.

[0042] The internal correctness signal described in step S4 is introduced into the lightweight probe model. This is used to estimate the probability that "if the answer is truncated and given in the current step, the final answer will be correct"; the probe output includes the logit value. and the corresponding confidence level ;

[0043] Formula (4): Probe mapping (to obtain logit)

[0044] (4)

[0045] Formula (5): Probe confidence (probability after Sigmoid)

[0046] (5)

[0047] in:

[0048] The probe model is typically implemented as a small multilayer perceptron (MLP) with parameter φ.

[0049] The meaning is the same as in formula (3), serving as the input feature of the probe;

[0050] This indicates that the probe is targeting the i-th step. The original scalar output (logit) has not yet been compressed to the [0,1] interval;

[0051] σ(·) represents the Sigmoid function, which maps any real number to the interval (0,1);

[0052] The probe confidence level for the i-th logical step can be interpreted as: if in this step... The thought process is immediately interrupted and the model is asked to provide a final answer, which is a probability estimate of the correctness of the final answer. This value is used as an internal correctness signal and participates in subsequent early stopping decisions.

[0053] This invention also provides a thought chain reasoning early termination control system based on entropy-probe dual signals, deployed on a server or local computer device, for implementing the method steps S1 to S7, including the following functional modules:

[0054] The thought chain sample collection module is used to perform complete thought chain reasoning on the training task set in the offline stage, and record the logical step division, output probability distribution, hidden state and standard answer.

[0055] Hidden State and Label Construction Module: Used to calculate hidden state features for each logical step, label the path correctness and reasoning completion status according to preset rules, and construct step-level training samples;

[0056] Probe training module: Used to train a lightweight probe model while freezing the parameters of the pre-trained language model, and obtain a decision function that maps the hidden state to the confidence of reasoning completion;

[0057] Threshold calibration module: Used to evaluate the impact of different combinations of entropy thresholds and confidence thresholds on accuracy and token consumption on the development set, and output parameter configurations for fast early stopping and stable early stopping;

[0058] Dual-signal early stop control module: Deployed in the online inference chain, it is used to calculate the step average entropy and probe confidence at the end of each logical step. Based on the fast early stop strategy, the stable early stop strategy and the fallback mechanism, it dynamically decides whether to continue generating the thought chain or terminate the inference and guide the model to give the answer.

[0059] In the offline phase, the thought chain sample acquisition module outputs raw records containing logical steps, probability distributions, hidden states, and standard answers. The hidden state and label construction module uses these records to form step-level features and reasoning completion / path correctness labels. The probe training module uses step-level samples to train and obtain a completion confidence decision function. The threshold calibration module jointly determines control parameters such as entropy threshold and confidence threshold on the development set. In the online phase, the dual-signal early-stop control module loads the probe and threshold configurations. At the end of each logical step, it calculates the average entropy and confidence of the step and performs a policy determination. If the early-stop condition is met, subsequent thought chain generation is terminated and the answer is guided to be output; otherwise, the generation of the next logical step continues.

[0060] Compared with existing technologies, the thought chain reasoning early stopping control method, computer device, computer-readable storage medium, and computer program product based on entropy-probe dual signals proposed in this invention have the following beneficial effects:

[0061] 1) By combining the external uncertainty signal of step average entropy and the internal correctness signal of probe confidence, the inherent defect of a single entropy signal that can only reflect subjective confidence and is difficult to identify the "confident but wrong" situation is compensated, thus improving the reliability of early stop decision-making.

[0062] 2) Without modifying the parameters of the pre-trained language model, this method alone can accelerate the inference stage of any large language model that supports the generation of thought chains, demonstrating good versatility and pluggability.

[0063] 3) Through the early stop controller, redundant thought chains can be aggressively cut off for simple samples, while conservative exit is maintained for complex samples. It automatically achieves adaptive control for samples of different difficulty levels, and significantly reduces the average number of reasoning tokens and reasoning latency while basically maintaining or even improving the question-level accuracy.

[0064] 4) The technical solution provided by this invention can effectively alleviate the problem of overthinking in large models in complex reasoning tasks. It can reduce reasoning costs and improve the stability of the reasoning process in various tasks such as mathematical reasoning, program generation, and logical reasoning. It provides a feasible engineering implementation path for the efficient deployment of large language models and large reasoning models. Attached Figure Description

[0065] Figure 1 This is a diagram of the online inference architecture of an embodiment of the present invention;

[0066] Figure 2 This is a comparison chart of experimental results of the embodiments of the present invention on different datasets and models;

[0067] Figure 3 This is a schematic diagram of the early stopping control system for the thought chain reasoning of the present invention. Detailed Implementation

[0068] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described are for illustrative purposes only and are not intended to limit the scope of the invention.

[0069] Example 1

[0070] An adaptive early stopping method for thought chain reasoning in large-scale inference models based on entropy-probe dual-signal fusion includes:

[0071] I. Offline Training Phase

[0072] Step 1: Generating Task Samples and Mind Chains

[0073] Obtain the dataset of reasoning tasks to be supported, such as a set of mathematical reasoning problems Q={q1,q2,…,qn}. Configure a unified thought chain prompt template for each problem, so that the pre-trained large language model can output the reasoning steps and the final answer step by step in the form of "1……2……3……" when given a problem.

[0074] During the offline phase, one or more complete reasoning operations are performed for each question in the dataset, retaining all the thought chain text generated by the model, and treating each numbered step as a "logical step". Simultaneously, the standard answer for each sample is recorded for subsequent path correctness annotation.

[0075] Step 2, Construction of Hidden States and Supervision Signals

[0076] During the complete inference process in step 1, at the end of each logical step, the corresponding intermediate hidden state is derived from the pre-trained language model. Specifically, the hidden vector at the end of the current step at the last token of the last layer of the decoder, or the average of the hidden vectors of all tokens in the current step, is selected as the internal representation vector ht for that step.

[0077] To train the probe, supervision labels need to be constructed for each step, using one or a combination of the following methods:

[0078] 1) Path correctness label: If the final answer of the complete thinking chain is consistent with the standard answer, then all steps in the thinking chain are marked as "leading to the correct answer"; if the final answer is wrong, then all steps in the thinking chain are marked as "leading to the wrong answer".

[0079] 2) Reasoning completion status label: Between multiple thought chains of the same question or within a single thought chain, by comparing the intermediate candidate answer with the standard answer, find the position where the correct answer first appears and is not overturned afterward, mark the step and several steps thereafter as "reasoning completed", and mark the previous steps as "reasoning incomplete".

[0080] This yields a step-level training sample set {(ht,yt)}, where ht represents the hidden state features and yt represents the binary labels, such as 1 indicating "sufficient to support the correct answer" and 0 indicating "reasoning has not yet been completed or is heading towards an error".

[0081] Step 3, Step-level probe model training

[0082] A lightweight probe model is constructed without modifying any parameters of the pre-trained language model. A one- or multi-layer fully connected network (MLP) is used to map the hidden state vector ht to a scalar score st, and then the probe confidence pt∈[0,1] is obtained by normalization using the Sigmoid function, which represents the probability estimate that "the current step is sufficient to support the correct answer".

[0083] The probe is trained using a binary cross-entropy loss function: yt is used as the supervision signal, and the sum or average of all training samples under this loss function is minimized. After training, fixed probe parameters are obtained, which are used in real-time to evaluate the internal correctness signal of each logical step during subsequent inference stages.

[0084] Step 4: Offline search and calibration of entropy threshold and probe confidence threshold

[0085] Metric calculation and data preparation: On the development set, the trained probe model is used to perform inference replay on the samples. For each generated logical step... Calculate the arithmetic mean of the Shannon entropies of all tokens in this step, and denote it as the step-average entropy. Simultaneously, the scalar score of the probe output is obtained as the probe confidence level. .

[0086] Grid Search and Early Stopping Simulation: Entropy threshold intervals [θent_min, θent_max] and confidence threshold intervals [θprob_min, θprob_max] are set, and a grid search strategy is used to scan different threshold combinations. For each threshold combination, the decision-making process of "stopping inference and outputting the answer once the condition is met" is simulated, and the overall accuracy and average token consumption under that combination are statistically analyzed.

[0087] Parameter combination selection: Among all candidate threshold combinations, their accuracy and average token consumption on the development set are compared simultaneously. The set of parameters with the highest accuracy and the lowest average token consumption among these high-accuracy combinations is selected as the final system configuration. The selected parameter set typically includes: an extremely low entropy threshold θent_fast and an extremely high confidence threshold θprob_fast for the fast early stopping channel; a stable confidence threshold θprob_stable for the stable early stopping channel; and a sliding window length L. This completes the offline training and parameter calibration phase.

[0088] II. Online Reasoning Stage

[0089] Step 5: Gradual Generation of the Thought Chain and Dual-Signal Calculation

[0090] During the deployment phase, for any user-inputted target question q, it is concatenated with a preset thought chain prompt template and then input into the pre-trained language model, so that the model generates reasoning steps step by step in the form of numbers.

[0091] When the model completes the output of the t-th logical step, perform the following operations:

[0092] 1) Calculate the step average entropy Ht based on the probability distribution of the tokens generated in this step, as an external uncertainty signal;

[0093] Token-level entropy

[0094] In this embodiment of the method, the uncertainty of the probability distribution of the large language reasoning model at the t-th generation position is expressed using Shannon entropy. To measure it, its definition is as follows:

[0095] Formula (1):

[0096] (1)

[0097] in:

[0098] t represents the index of the t-th token currently generated;

[0099] V represents the vocabulary set output by the model;

[0100] v represents any candidate token in the vocabulary V;

[0101] This represents the conditional probability that "the next token will have a value of v" at position t. = ( = v | , q);

[0102] log represents the logarithmic function, and the logarithm with the natural logarithm as the base is usually chosen;

[0103] Let be the Shannon entropy at position t. A larger value indicates a more dispersed probability distribution at that location and a more hesitant model. The smaller the value, the more concentrated the distribution and the more confident the model.

[0104] Step average entropy - external uncertainty signal

[0105] To characterize the external uncertainties in the reasoning process at the granularity of logical steps, step-average entropy is introduced. , is used to describe the overall entropy level of the i-th logical step.

[0106] Formula (2):

[0107] (2)

[0108] in:

[0109] This represents the i-th logical step, such as a whole paragraph of reasoning text in the format “(1)…(2)…(3)…”;

[0110] T( ) represents the set of token location indices included in step S_i;

[0111] |T( )| represents the size of set T(S_i), i.e., the number of steps. The number of tokens in the middle;

[0112] Let be the Shannon entropy at position t, with the same meaning as in formula (1);

[0113] For steps The average entropy, equal to the arithmetic mean of the entropy values ​​of all tokens within that step, is used as an external uncertainty signal for that step.

[0114] 2) Derive the hidden state vector ht corresponding to this step from the language model, input it into the probe model, and obtain the probe confidence pt as the internal correctness signal;

[0115] III. Step-level Hidden States - Internal Representation

[0116] To characterize the model's internal "thought state" at the logical step level, the hidden vectors within that step are averaged to obtain the step-level hidden state. ;

[0117] Formula (3):

[0118] (3)

[0119] in:

[0120] ∈ R^d represents the last hidden vector representation of the large language reasoning model at position t, with dimension d;

[0121] T( ),|T( The meaning of )| is the same as in formula (2);

[0122] Indicate steps The step-level hidden state is the hidden vector of all tokens within the step. The vector obtained by taking the arithmetic mean can be understood as the overall internal representation of the model after completing this step of reasoning.

[0123] IV. Probe Output and Probe Confidence - Internal Correctness Signal

[0124] Based on the aforementioned step-level hidden states, a lightweight probe model is introduced. This is used to estimate the probability that "if the answer is truncated and given in the current step, the final answer will be correct." The probe output includes the logit value z_i and the corresponding confidence level. .

[0125] Formula (4): Probe mapping (to obtain logit)

[0126] (4)

[0127] Formula (5): Probe confidence (probability after Sigmoid)

[0128] (5)

[0129] in:

[0130] The probe model is typically implemented as a small multilayer perceptron (MLP) with parameter φ.

[0131] The meaning is the same as in formula (3), serving as the input feature of the probe;

[0132] This indicates that the probe outputs the original scalar (logit) for the i-th step S_i, which has not yet been compressed to the [0,1] interval;

[0133] σ(·) represents the Sigmoid function, which maps any real number to the interval (0,1);

[0134] The probe confidence level for the i-th logical step can be interpreted as: if in this step... The thought process is immediately interrupted, and the model is asked to provide a final answer, which is a probability estimate of the correctness of the answer. This value is used as an internal correctness signal and participates in subsequent early stopping decisions.

[0135] 3) Input Ht and pt into the early stop controller for decision-making.

[0136] Step 6, Rapid Early Stop Control

[0137] If the current step satisfies any or all of the following conditions: – The average entropy of the step, Ht, is lower than the extremely low entropy threshold, θent_fast; – The probe confidence, pt, is higher than the extremely high confidence threshold, θprob_fast.

[0138] If the model is considered to have a very high degree of confidence in the current reasoning path, the early stop controller will no longer allow the generation of subsequent thought chains. Instead, it will append a "Please give the final answer directly" output instruction to the current context, guiding the model to output the final answer within a small number of subsequent tokens. This approach maximizes the elimination of redundant reasoning steps while maintaining a high success rate.

[0139] Step 7, Stabilize Early Stop Control

[0140] When the current step does not trigger a fast early stop, add Ht and pt to a sliding window of fixed length L to maintain the dual-signal sequence of the most recent L logical steps.

[0141] If the probe confidence of all steps within the window is not lower than the stability threshold θprob_stable, and the average entropy of the steps is generally at a low to medium level (e.g., the average window entropy is lower than a certain empirical value), then the model is considered to have maintained high confidence and has not shown any obvious confusion in the recent inference process. In this case, the current step can be regarded as a candidate inference completion point, triggering stable early stopping control.

[0142] In implementation, a lightweight check can be added before triggering a stable early stop. For example, add a sentence to the model: "Please summarize the final answer directly based on the above reasoning." Check whether the answer is consistent with the candidate answer automatically extracted in the window. If the check passes, the subsequent thought chain generation is terminated and the answer is retained. If the check fails, the early stop is abandoned and the next logical step is generated.

[0143] Step 8, Complete Mindset Safety Net Mechanism

[0144] During the inference process, the number of generated thought chain tokens is continuously accumulated. When the number of tokens reaches the preset budget limit or the number of logical steps exceeds the maximum allowed value, if a fast early stop or stable early stop is not triggered, the model will default to completing the output of a thought chain of preset length and then directly providing the answer. This ensures that even on a very small number of abnormal samples, the inference process can safely terminate within bounded resources.

[0145] Example 2

[0146] This embodiment selects DeepSeek-7B-Instruct as the pre-trained large language model base and MATH-500 (i.e., 500 test questions randomly sampled from the MATH dataset covering algebra, geometry, number theory, and other fields) as the validation dataset, and demonstrates in detail the specific implementation process of the thought chain reasoning early stopping control method based on entropy-probe dual signals described in this invention.

[0147] I. Offline Preparation and Training Phase

[0148] Step S1: Task Acquisition and Formatting. Select the MATH-500 dataset as the training and testing source. Configure the thought chain prompt template (Prompt Template), setting the prompt to: "Please reason step by step and put your final answer within \boxed{}. \n Question: [Question Text]". Set the logical step boundary recognition rule: using the newline character \n as a natural segmentation point, segment the continuous text generated by DeepSeek-7B into sequential logical steps. .

[0149] Step S2: Sample Construction and Labeling. Input the training subset from MATH-500 into the DeepSeek-7B model for greedy decoding to generate a complete thought chain. For example, for an algebra problem: "If f(x) = 2x + 3, find f(f(1))". The model generates the following complete thought chain:

[0150] : First, calculate the inner function value f(1) = 2(1) + 3 = 5.

[0151] : Next, use this result to evaluate the outer function f(5).

[0152] : f(5) = 2(5) + 3 = 10 + 3 = 13.

[0153] : The calculation is verified.

[0154] The final answer is boxed {13}. Record each step. The corresponding Logits distribution. Labeling process: Extract the standard answer "13". The system automatically backtracks and checks, discovering... The steps already contain the correct answer "13" and are logically complete, therefore $S_3, S_4, S_5$ are marked as "reasoning complete (positive example, Label=1)". Mark as "Incomplete status (negative example, Label=0)".

[0155] Step S3: Probe training extracts the last hidden state vector of the token at the end of each logical step in the DeepSeek-7B model. (The hidden layer dimension of DeepSeek-7B is d=4096). A lightweight MLP probe network is constructed with the structure Linear(4096→256) -> ReLU -> Linear(256→1) -> Sigmoid. The probes are trained using the {(hidden state, completed state label)} data constructed in step S2 for binary classification, with the optimization objective being to minimize the binary cross-entropy loss. The probe parameters are frozen after training. .

[0156] II. In the online reasoning stage, let's take a specific test question as an example.

[0157] Step S4: Target Input and Dual-Signal Real-Time Calculation Input Test Question: "Solve for x: 3x - 7 = 11". Generate Step 1 ( ): "Add 7 to both sides of the equation: 3x = 18."

[0158] Calculate the external signal, Formula 2: Statistical The entropy value of each token is used to calculate the average entropy of each step. = 0.45 (a relatively high value, indicating that the model exhibits some divergence when constructing equation transformations).

[0159] Calculate the internal signal, formula 3-5: Extract The probe is input with the hidden state at the end, and the confidence level is output. = 0.20 (indicating no answer has been received yet). Generate step 2. "Divide by 3: x = 6." Calculate the external signal: This step has simple logic and high model confidence, and the calculation yields... = 0.08 (extremely low entropy, very confident). Calculate the internal signal: Extract... The probe is input with the hidden state at the end, and the confidence level is output. = 0.99, the probe strongly senses that the answer has been generated.

[0160] Step S5: Determine the rapid early stopping strategy. Preset parameters: extremely low entropy threshold. = 0.15, extremely high confidence threshold = 0.95. In At the end: =0.45 > 0.15, and =0.20 < 0.95, early stopping is not triggered. At the end: =0.08 < 0.15, and =0.99 > 0.95. Judgment result: The dual-signal extreme value condition is met simultaneously, and the model is determined to be in a state of extremely high confidence, triggering rapid early stopping.

[0161] Step S6: Stabilize the early stopping strategy. In this example, it has been truncated by S5. If it were not truncated, the following would be executed: if the above... If the probe score is 0.85 (below the fast threshold), then the system enters the sliding window, with a window length L=2. The system will check [ , Sequence, due to The confidence level is too low and the stable window condition is not met, so generation will continue. .

[0162] Step S7: Termination and Output Because a rapid early stop was triggered in step S5, the system immediately discards redundant verification steps that the original model might continue to generate, such as "Let me double check...", "Substituting x back...", etc., and directly forces truncation. System Operation: In The text is appended with the hint "The final answer is", which guides the model to output the final answer\boxed{6}.

[0163] Comparison of implementation effect data

[0164] On the MATH-500 test set, compare the "original DeepSeek-7B thought process" with the "DeepSeek-7B using the method of this embodiment":

[0165] Accuracy: The original model achieved 90.8%, while the method improved it to 91.2%. The accuracy did not decrease, and it even slightly improved due to the reduction of correct errors caused by model self-doubt.

[0166] Average Token Consumption: The original reasoning method consumed an average of 3661 tokens per question, which was reduced to 2193 tokens per question after applying this method.

[0167] Compression rate: It achieved a compression rate of approximately 59.9%, which means that about half of the computing resources were saved.

[0168] In summary, this embodiment verifies that the entropy-probe dual-signal mechanism can effectively identify high-confidence nodes such as "x=6" in the MATH-500 problem on the DeepSeek-7B model, successfully terminating the reasoning at the first moment of logical closure, and achieving a win-win situation of reasoning acceleration and accuracy.

[0169] Example 3

[0170] like Figure 3 As shown, a thought chain reasoning early termination control system based on entropy-probe dual signals, deployed on a server or local computer device, includes the following functional modules:

[0171] 1) Mind Chain Sample Acquisition Module: Used to perform complete mind chain reasoning on the training task set in the offline stage, and record the logical step division, output probability distribution, hidden state and standard answer.

[0172] 2) Hidden State and Label Construction Module: This module is used to calculate the hidden state features for each logical step, label the path correctness and reasoning completion status according to preset rules, and construct step-level training samples.

[0173] 3) Probe training module: Used to train a lightweight probe model with the parameters of the pre-trained language model frozen, so as to obtain a decision function that maps the hidden state to the confidence of reasoning completion.

[0174] 4) Threshold calibration module: Used to evaluate the impact of different combinations of entropy thresholds and confidence thresholds on accuracy and token consumption on the development set, and output parameter configurations for fast early stopping and stable early stopping.

[0175] 5) Dual-signal early stop control module: Deployed in the online inference chain, it is used to calculate the step average entropy and probe confidence at the end of each logical step. Based on the fast early stop strategy, the stable early stop strategy and the fallback mechanism, it dynamically decides whether to continue generating the thought chain or terminate the inference and guide the model to give the answer.

[0176] In the offline phase, the thought chain sample acquisition module generates and records the logical steps, probability distribution, hidden states, and standard answers of the complete thought chain; the hidden state and label construction module converts the records into step-level features and labels such as "complete / correct"; the probe training module trains a lightweight probe while freezing the parameters of the large model to obtain the mapping of "hidden state → completion confidence"; the threshold calibration module jointly searches for entropy thresholds and confidence thresholds on the development set and outputs the parameter configurations for fast early stopping and stable early stopping. In the online phase, the dual-signal early stopping control module calculates the step average entropy and probe confidence at the end of each logical step, and decides whether to continue generation or terminate early according to the fast strategy, stable strategy, and fallback mechanism, and guides the model to directly output the answer.

[0177] Figure 2 These are experimental results on multiple publicly available datasets, including GMS8k (elementary level difficulty), MATH-500 (middle school level difficulty), and AIME24 and 25 (competition level difficulty). These datasets contain math and logic reasoning problems. Figure 2 As shown, our method was tested on multiple models, such as Deepseek's 7B and 14B, and qwen3-8b, and achieved better accuracy and compression ratio (CR) than similar methods. Accuracy represents the proportion of correct answers, and compression ratio represents the number of tokens consumed. The lower the compression ratio, the better the model performs.

[0178] Through the above embodiments, those skilled in the art can realize the early stopping control method and system for thought chain reasoning based on entropy-probe dual signals, and integrate the early stopping control capability described in this invention into the existing large language model reasoning framework without creative effort.

Claims

1. A method for controlling early stopping of thought chain reasoning based on entropy-probe dual signals, comprising the following steps: Offline preparation stage Step S1: Obtain the set of reasoning tasks to be processed and the large-scale language reasoning model, set the thought chain generation format and logic step boundary recognition rules, so that the model outputs the thought chain text step by step in the numbering format during the reasoning process, and can detect the boundary of the current logic step at the end of each numbered step. Step S2: During the training phase, the task set is input into the pre-trained language model to generate a complete thought chain reasoning process, record the output probability distribution and intermediate hidden states corresponding to each logical step, and automatically label the path correctness information and reasoning completion status information of each logical step based on the thought chain generation format and logical step boundary recognition rules to construct a step-level training sample set. Step S3: Extract hidden state signals for each logic step to obtain step-level hidden states; use whether the current state is a reasoning completion state as a label to train the probe, and output the confidence score of whether the current logic step is "sufficient to support the correct answer" based on the step-level hidden states. Online reasoning stage Step S4: Input the target problem into the pre-trained language model and generate logical steps step by step under the set thought chain prompts; at the end of each logical step, calculate the average entropy of the step within that step as an external uncertainty signal, and extract the corresponding step-level hidden state and input it into the pre-trained probe model to obtain the probe confidence of the current logical step as an internal correctness signal. Step S5: Set up a fast early stop strategy: When the average entropy of the current logical step is lower than the preset very low entropy threshold or the probe confidence is higher than the preset very high confidence threshold, the model is determined to be in a very high confidence state, and early stop control is directly triggered to terminate the generation of subsequent thought chains and guide the model to output the final answer. Step S6: When the fast early stopping condition is not met, perform joint judgment on several consecutive logical steps: maintain a sliding window of fixed length, collect the step average entropy and probe confidence of each logical step in the window, when the probe confidence of all logical steps in the window is not lower than the preset stability threshold and the overall step average entropy is at a medium-low level, the current logical step is regarded as a candidate early stopping point, perform lightweight consistency verification to confirm whether the reasoning result is stable, trigger stable early stopping control and output the final answer when the verification passes, and abandon this early stopping when the verification fails and continue to generate subsequent logical steps; Step S7: During the reasoning process, continuously monitor the number of generated thought chain tokens and the preset token budget or maximum thought chain length. When the budget limit or maximum length is reached but early stop control is not triggered, generate a complete thought chain by default and output the final answer, so that the entire reasoning process is safely terminated under the given resource constraints.

2. The method for controlling early stopping of thought chain reasoning based on entropy-probe dual signals according to claim 1, characterized in that, The underlying token-level entropy of the external uncertainty is the Shannon entropy H_t, which measures the probability distribution uncertainty of the large language reasoning model at the t-th generation position. Its definition is as follows: Formula (1): （1）； in: t represents the position index of the t-th token currently generated; V represents the vocabulary set output by the model; v represents any candidate token in the vocabulary V; This represents the conditional probability that "the next token will have a value of v" at position t. = ( = v | , q); log represents the logarithmic function, and the logarithm with the natural logarithm as the base is usually chosen; Let be the Shannon entropy at position t. A larger value indicates a more dispersed probability distribution at that position and a more hesitant model. The smaller the value, the more concentrated the distribution and the more confident the model.

3. The method for controlling early stopping of thought chain reasoning based on entropy-probe dual signals according to claim 1, characterized in that, The external uncertainty signal introduces the step average entropy , used to describe the overall entropy level of the i-th logical step; Formula (2): （2）； in: This represents the i-th logical step, i.e., a complete piece of reasoning text in the format "1…2…3…". T( () indicates steps The set of token location indices contained therein; |T( )| represents set T( The size of ) is the step The number of tokens in the middle; Let be the Shannon entropy at position t, with the same meaning as in formula (1); For steps The average entropy, equal to the arithmetic mean of the entropy values ​​of all tokens within that step, is used as an external uncertainty signal for that step.

4. The method for controlling early stopping of thought chain reasoning based on entropy-probe dual signals according to claim 1, characterized in that, The step-level hidden state is obtained by averaging the hidden vectors in step S4. , Formula (3): （3）； in: ∈ The last hidden vector representation of the large language reasoning model at position t has a dimension of d. T( ),|T( The meaning of )| is the same as in formula (2); Indicate steps The step-level hidden state is the hidden vector of all tokens within the step. The vector obtained by taking the arithmetic mean can be understood as the overall internal representation of the model after completing this step of reasoning.

5. The method for controlling early stopping of thought chain reasoning based on entropy-probe dual signals according to claim 1, characterized in that, The internal correctness signal is introduced into a lightweight probe model. This is used to estimate the probability that "if the answer is truncated and given in the current step, the final answer will be correct"; the probe output includes the logit value. and the corresponding confidence level ; Formula (4): Probe mapping, to obtain logit （4）； Formula (5): Probe confidence, probability after Sigmoid function （5）； in: The probe model is typically implemented as a small multilayer perceptron (MLP) with parameter φ. The meaning is the same as in formula (3), serving as the input feature of the probe; This indicates that the probe is targeting the i-th step. The original scalar output (logit) has not yet been compressed to the [0,1] interval; σ(·) represents the Sigmoid function, which maps any real number to the interval (0,1); The probe confidence level for the i-th logical step can be interpreted as: if in this step... The thought process is immediately interrupted and the model is asked to provide a final answer, which is a probability estimate of the correctness of the final answer. This value is used as an internal correctness signal and participates in subsequent early stopping decisions.

6. A thought chain reasoning early termination control system based on entropy-probe dual signals, deployed on a server or local computer device, includes the following functional modules: The thought chain sample collection module is used to perform complete thought chain reasoning on the training task set in the offline stage, and record the logical step division, output probability distribution, hidden state and standard answer. Hidden State and Label Construction Module: Used to calculate hidden state features for each logical step, label the path correctness and reasoning completion status according to preset rules, and construct step-level training samples; Probe training module: Used to train a lightweight probe model while freezing the parameters of the pre-trained language model, and obtain a decision function that maps the hidden state to the confidence of reasoning completion; Threshold calibration module: Used to evaluate the impact of different combinations of entropy thresholds and confidence thresholds on accuracy and token consumption on the development set, and output parameter configurations for fast early stopping and stable early stopping; Dual-signal early stop control module: Deployed in the online inference chain, it is used to calculate the step average entropy and probe confidence at the end of each logical step. Based on the fast early stop strategy, the stable early stop strategy and the fallback mechanism, it dynamically decides whether to continue generating the thought chain or terminate the inference and guide the model to give the answer.

Images