A reasoning control method and system for a large language model thinking chain

Abstract

This invention relates to the field of natural language processing technology and discloses a method and system for inference control of the thought chain in a large language model. The method utilizes a sparse autoencoder to decompose the hidden layer representation of a large language model into a higher-dimensional sparse atomic space, thereby constructing a reversible mapping from dense representation to sparse atomic representation. Statistical analysis is performed on comparative datasets with different inference depths to identify and classify extended thought atoms that promote inference expansion and convergent thought atoms that promote inference convergence. The text of the input question is acquired, and the complexity of the input question is evaluated based on the activation intensity of the extended thought atoms identified from the input question. This complexity is then mapped to an inference control strength. An intervention vector is generated based on the inference control strength and the thought atoms in the input question. The intervention vector is applied to the intermediate hidden layer representation of the model to control the inference depth of the large language model. This invention improves the accuracy of inference depth control for large language models.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a reasoning control method and system for a large language model thought chain. Background Technology

[0002] Chain-of-Thought (CoT) reasoning, a crucial technique for enhancing the reasoning capabilities of large language models, improves model performance on complex tasks such as mathematical reasoning, logical judgment, and common-sense reasoning by breaking down complex problems into multiple intermediate steps. From early Zero-shot CoT and Few-shot CoT to current self-consistent and tool-enhanced CoT, this technique has become a key pathway for improving the capabilities of large language models. In particular, models such as GPT-4, Claude, and o1 have achieved significant results in solving complex problems through deep chain-of-thought reasoning.

[0003] However, controlling the depth of thought, or the length of reasoning, during the thought chain reasoning process remains a technical challenge in current applications of large language models. Problems of different complexities require varying depths of reasoning; simple fact queries may require fewer reasoning steps, while complex mathematical proofs may require more. Current reasoning models often employ similar reasoning patterns when handling problems of different complexities, potentially leading to lengthy reasoning for simple problems and insufficient depth of reasoning for complex ones. Therefore, behavioral control techniques for the model's thought chain are crucial for ensuring the controllability of the thinking process in large language models.

[0004] Current mainstream control methods include two main categories: cue word engineering and vector intervention. Cue word engineering influences model behavior by adding guiding text to the input, but it suffers from problems such as high design difficulty and limited stability of results. Vector intervention methods, such as contrastive activation addition, generate intervention vectors by calculating the difference in hidden layer representations between positive and negative samples, directly modifying the internal calculation process of the model. This achieves more precise behavior control to a certain extent, but finding a suitable vector module remains a technical challenge. Summary of the Invention

[0005] To address the technical problem of limited precision in reasoning depth control in current large language model thinking chain reasoning, which leads to poor model response quality, this invention provides a reasoning control method and system for large language model thinking chains.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] This invention discloses a reasoning control method for a large language model thinking chain, comprising the following steps, namely S1 to S5.

[0008] S1. By using a sparse autoencoder, the hidden layer representation of a large language model is decomposed into a higher-dimensional sparse atomic space, thereby constructing an invertible mapping from dense representation to sparse atomic representation.

[0009] S2. Perform statistical analysis on comparative datasets with different inference depths to identify and classify extended thinking atoms that promote inference expansion and convergent thinking atoms that promote inference convergence.

[0010] S3. Obtain the text of the input question, assess the complexity of the input question based on the activation intensity of the extended thinking atoms identified from the input question, and map the complexity to the reasoning control strength.

[0011] S4. Generate an intervention vector based on the inference control strength and the thought atoms in the input question.

[0012] S5. The intervention vector is applied to the intermediate hidden layer representation of the large language model to control the inference depth of the large language model and guide the large language model to adaptively generate answer text according to the complexity of the input question.

[0013] As a further improvement to the above scheme, step S1 includes the following specific steps, namely S11~S12.

[0014] S11. Through the encoding process of a sparse autoencoder, the hidden layer representation of the large language model is projected onto a high-dimensional sparse atomic space, expressed as:

[0015] ;

[0016] In the formula, This is the sparse representation after encoding; This represents the hidden layer representation of a large language model. , R For the set of real numbers, For sequence length, For hidden layer dimensions; For encoding the weight matrix, For atomic space dimension, , The expansion coefficient is used to balance expressive power and sparsity; To encode the bias vector, it is extended through a broadcast mechanism. Add the dimensions together; This is the activation function.

[0017] S12. Reconstruct the original representation from the encoded sparse representation using a decoder, as shown in the expression:

[0018] ;

[0019] In the formula, for The result after reconstructing the original representation; To decode the weight matrix; This is the decoding bias vector.

[0020] As a further improvement to the above scheme, the sparse autoencoder is optimized by minimizing the combination loss function, expressed as:

[0021] ;

[0022] In the formula, This is the combined loss; Represents the L2 norm. Represents the L1 norm; To reconstruct the error term; This is a sparsity constraint term; These are the sparsity weighting coefficients. Training was performed using the AdamW optimizer, with a learning rate set to [value missing]. Weight decay is set to The batch size was set to 128, the gradient accumulation steps were set to 4, and the convergence condition was set to the validation set reconstruction error being less than 5% and no improvement for 5 consecutive training cycles.

[0023] As a further improvement to the above scheme, step S2 includes the following specific steps, namely S21 to S24.

[0024] S21. Construct a triplet comparison dataset ;in, Enter the query; For a concise answer; For detailed reasoning and answering; superscript Indicates a concise reasoning pattern, superscript Indicates a detailed reasoning pattern; This represents the number of samples.

[0025] S22. For each thought atom Calculate its activation statistics under different inference modes; where the statistical parameters for the succinct mode are:

[0026] ;

[0027] ;

[0028] The statistical parameters in the detailed inference mode are:

[0029] ;

[0030] ;

[0031] In the formula, For the atom of thought Average activation strength in concise mode; For the atom of thought Average activation intensity in detailed reasoning mode; Representing the atom of thought In response to the problem Generate answer The average activation intensity during the process; Representing the atom of thought The standard deviation of activation intensity in concise reasoning mode; Representing the atom of thought The standard deviation of activation intensity in detailed reasoning mode.

[0032] S23. Based on the activation statistic calculated in step S22, the standardized effect size is used to quantify the activation differences of thought atoms, expressed as:

[0033] ;

[0034] In the formula, Indicates the first The effect size of a single thought atom.

[0035] S24. Classify thought atoms based on effect size, dividing them into extended thought atoms or convergent thought atoms; where, for extended thought atoms:

[0036] ;

[0037] For convergent thinking atoms:

[0038] ;

[0039] in, The set standard threshold; For extended thinking atomic sets; It is a convergent thinking atom set.

[0040] As a further improvement to the above scheme, step S3 includes the following specific steps, namely S31~S32.

[0041] S31. Obtain the input problem. Define the complexity of the input problem based on the activation intensity of the extended thinking atoms. The expression is:

[0042] ;

[0043] In the formula, The input query is the current input question. In the atom of thought The activation value on; Weights are based on effect size; This is a sensitivity parameter that controls the steepness of the sigmoid function; The median complexity of the training set is obtained by using the training set. calculate:

[0044] The sigmoid function is used to ensure ; To take the median.

[0045] S32. Map complexity to inference control strength, expressed as:

[0046] ;

[0047] In the formula, For reasoning control strength; The basic control intensity, with a value range of [value range missing]. ; For the maximum control strength, the value range is: .

[0048] As a further improvement to the above scheme, step S4 includes the following specific steps, namely S41 to S43.

[0049] S41. Extract semantic vectors from the model word embedding space and construct a bidirectional control vector, expressed as:

[0050] ;

[0051] ;

[0052] In the formula, For extended direction control vector; This is the convergence direction control vector; For the first The word embedding vector corresponding to each thought atom.

[0053] S42. Inference-based control strength Generate hybrid control vectors :

[0054] ;

[0055] S43. Generate intervention vectors :

[0056] ;

[0057] In the formula, This is the intervention intensity coefficient, used to control the degree of influence of the vector in the hidden space.

[0058] As a further improvement to the above scheme, in step S41, the bidirectional control vectors are further subjected to L2 normalization, and the expression is:

[0059] ;

[0060] ;

[0061] In the formula, This represents the L2 norm.

[0062] In step S43, the intervention vector is also standardized, and the expression is:

[0063] ;

[0064] In the formula, For standardized strength.

[0065] As a further improvement to the above scheme, step S5 includes the following specific steps, namely S51~S52.

[0066] S51. Select the intermediate layer of the large language model as the target intervention layer, the expression is:

[0067] ;

[0068] In the formula, The layer index represents the target intervention layer; This is the layer position coefficient, used to determine the relative location of the intervention; This represents the total number of layers in the large language model. This indicates rounding down to the nearest integer.

[0069] S52. Apply the intervention vector at the target intervention layer, expressed as:

[0070] ;

[0071] In the formula, For the original number Hidden layer representation, For the revised first Hidden layer representation; The intervention intensity is determined based on inference control intensity, and the expression is:

[0072] ;

[0073] In the formula, The intervention amplification factor converts the inference control strength into the hidden layer intervention strength.

[0074] As a further improvement to the above scheme, the intervention amplification factor is set to The adaptive residual connection is used for smooth transition, as follows:

[0075] ;

[0076] In the formula, For the first Hidden layer representation; the final intervention result; The mixing coefficient, min(·) selects the minimum value, and max(·) selects the maximum value.

[0077] Following step S52, the final intervention result is also evaluated. Perform layer normalization.

[0078] This invention also discloses a reasoning control system for a large language model thinking chain, which applies the reasoning control method for a large language model thinking chain as described above; the reasoning control system includes: a sparse atom representation learning module, a thinking atom recognition and classification module, a complexity perception mechanism module, an intervention vector generation module, and a reasoning process intervention control module.

[0079] The sparse atom representation learning module is used to decompose the hidden layer representation of a large language model into a higher-dimensional sparse atom space using a sparse autoencoder, thereby constructing an invertible mapping from dense representation to sparse atom representation.

[0080] The Mind Atom Recognition and Classification Module is used to perform statistical analysis on comparative datasets with different inference depths, thereby identifying and classifying extended mind atoms that promote inference expansion and convergent mind atoms that promote inference convergence.

[0081] The complexity-aware mechanism module is used to acquire the text of the input question, evaluate the complexity of the input question based on the activation intensity of the extended thinking atoms identified from the input question, and map the complexity to the inference control strength.

[0082] The intervention vector generation module is used to generate an intervention vector based on the inference control strength and the thought atoms in the input question.

[0083] The reasoning process intervention control module is used to apply the intervention vector to the intermediate hidden layer representation of the large language model in order to control the reasoning depth of the large language model and guide the large language model to adaptively generate answer text according to the complexity of the input question.

[0084] Compared with the prior art, the beneficial effects of the present invention are:

[0085] 1. This invention discloses a reasoning control method for the thought chain of a large language model, proposing a reasoning depth control mechanism based on sparse atomic representation. This mechanism projects the high-dimensional hidden layer representation of the large language model onto an interpretable atomic space using a sparse autoencoder, identifying key thought atoms that promote reasoning expansion and convergence. By comparing and analyzing the differences in atomic activation at different reasoning depths, this mechanism achieves a precise mapping from macroscopic reasoning behavior to microscopic semantic units. This "atomic-level" fine control avoids interference with other basic language understanding and logical functions of the model, thus maintaining reasoning quality and semantic coherence while precisely controlling the reasoning length of the large language model.

[0086] 2. This invention constructs an adaptive inference length adjustment mechanism based on problem complexity awareness. It can automatically evaluate the inference requirements of the input problem according to the activation mode of the extended thinking atoms. It establishes an adaptive conversion mechanism from complexity to inference control intensity through linear mapping, which solves the limitations of needing to manually preset inference depth parameters and lacking adaptive capabilities, and improves the system's flexibility and control accuracy in handling problems of different difficulties.

[0087] Based on this, for large language model application systems, when a simple problem is detected, the method of this invention will guide the model to "converge quickly", significantly reducing unnecessary inference steps and computational overhead, and saving computational resources; while when a complex problem is detected, the method of this invention will promote the model to "fully unfold", ensuring that the system invests sufficient computational resources to prevent a decline in inference quality.

[0088] 3. This invention establishes a reasoning process control framework based on hidden layer intervention. By applying control vectors to the intermediate layers of the model, combined with adaptive residual connections and layer normalization mechanisms, it achieves precise length control while maintaining the integrity of reasoning logic and semantic coherence, providing a technical solution for reasoning depth control of large language models in practical applications. Attached Figure Description

[0089] Figure 1 This is a flowchart of the reasoning control method for the large language model thinking chain in Embodiment 1 of the present invention.

[0090] Figure 2 This is an example diagram illustrating different reasoning patterns for a simple problem in Embodiment 1 of the present invention.

[0091] Figure 3 This is a schematic diagram illustrating the principle of constructing an invertible mapping using a sparse autoencoder in Embodiment 1 of the present invention.

[0092] Figure 4 This is a schematic diagram illustrating the principle of identifying extended thinking atoms and convergent thinking atoms in Embodiment 1 of the present invention.

[0093] Figure 5This is a schematic diagram illustrating the principle of intervention and control of the reasoning process in Embodiment 1 of the present invention.

[0094] Figure 6 This is a framework diagram of the reasoning control system of the large language model thinking chain in Embodiment 2 of the present invention. Detailed Implementation

[0095] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0096] Example 1

[0097] Existing thought chain length control technologies have the following limitations:

[0098] (1) Existing control methods have coarse control granularity and lack atomic-level precision manipulation capabilities. Existing vector intervention methods, such as Contrastive Activation Addition (CAA), use holistic representation difference calculation, mixing multiple semantic concepts and cognitive functions in the same control vector. This makes it impossible to accurately identify and independently manipulate specific neuronal components that control the depth of reasoning. This coarse-grained control is prone to side effects such as semantic drift and logical breaks when adjusting the length of reasoning, making it difficult to achieve refined and controllable manipulation of the reasoning process.

[0099] (2) Existing technologies are lacking in adaptive adjustment, making it difficult to adjust the inference depth based on the complexity characteristics of the input problem. Current technologies mainly rely on fixed parameters or simple heuristic rules for inference depth control, lacking the ability to automatically perceive and evaluate the complexity characteristics of the input problem. This results in the system being unable to differentiate between problems of different complexities: simple problems are over-expanded, leading to a waste of computational resources, while complex problems are not adequately processed, resulting in a decline in inference quality.

[0100] (3) Existing technologies have limited control accuracy and stability. Existing methods have low accuracy in inference length control and are prone to problems such as inference truncation, redundant expansion or control failure. At the same time, they lack effective stability guarantee mechanisms, and the control process may cause abnormal fluctuations in the internal representation of the model, affecting inference quality and semantic coherence.

[0101] To address the limitations of existing technologies, sparse autoencoders (SAEs) offer a solution to some extent. SAEs have made progress in neural network interpretability research in recent years. By projecting dense neural network representations onto a higher-dimensional sparse space, SAEs decompose ambiguous neurons into relatively unambiguous atomic features, thus solving the problem of single neurons encoding multiple semantic concepts in traditional neural networks to some extent. Related research shows that SAEs can identify neuronal atoms corresponding to specific concepts, such as security and honesty, providing a technical foundation for model behavior control.

[0102] This invention, combining SAE technology, addresses the pain points of existing thought chain length control technologies by providing a reasoning control method for large language model thought chains. This reasoning control method can be applied to question-answering systems in fields such as teaching, aiming to solve the following core technical problems:

[0103] 1. Precise control problem: How to use sparse representation learning techniques to decompose the high-dimensional hidden layer representation of a large language model into an interpretable space of thought atoms, accurately identify the neuronal atoms that control the depth of reasoning, especially to distinguish and independently manipulate the extended atoms that promote deep thinking and the convergent atoms that promote rapid conclusions, so as to achieve atomic-level fine control over the reasoning process.

[0104] 2. Complexity Adaptive Awareness Problem: How to construct an automatic awareness and evaluation mechanism based on problem complexity, so that the model can intelligently identify the reasoning depth requirements of different types of problems, automatically determine the appropriate target reasoning length, and achieve adaptive reasoning depth adjustment from rapid convergence of simple problems to full expansion of complex problems.

[0105] 3. Control precision and stability issues: How to design a hidden layer intervention strategy and stability guarantee mechanism based on control vectors to maintain inference quality and semantic coherence while achieving high-precision inference length control, and ensure that the control process does not negatively affect the model's basic language understanding and generation capabilities.

[0106] The reasoning control method of the large language model thinking chain in this embodiment decomposes the internal hidden layer representation of the large language model into an interpretable atomic space through a sparse autoencoder. It uses comparative statistical analysis to accurately identify key thinking atoms related to reasoning expansion and convergence. Combined with a problem complexity perception mechanism and control vector generation, it achieves refined and intelligent control over the reasoning depth of the model.

[0107] Specifically, please refer to Figure 1 The reasoning control method includes the following steps, namely S1 to S5.

[0108] S1. By utilizing a sparse autoencoder, the hidden layer representation of a large language model is decomposed into a higher-dimensional sparse atomic space, thereby constructing an invertible mapping from dense representation to sparse atomic representation, laying the foundation for subsequent precise manipulation, such as... Figure 3 As shown.

[0109] Figure 3 The following is an example of a triplet:

[0110] Question: If the area of ​​a square is 36 square centimeters, what is its perimeter?

[0111] Short answer: According to the formula for the area of ​​a square, the side length... The circumference is 4 x 6 = 24 centimeters.

[0112] Detailed reasoning: To solve this problem, we need to apply the basic geometric properties and relevant formulas of a square. The first step is to use the area formula to find the side length: the area formula for a square is S = a... 2 Where 'a' is the side length; Substitute the known data: a 2 =36, solve this linear equation: a = 6 cm. We can verify this result, 6 2 =36. The second step is to use the perimeter formula to find the perimeter: The perimeter formula for a square is P=4a. Since all four sides are equal in length, substituting this into the formula, we get the side length: P=4×6=24 cm. Additional explanation: This type of problem demonstrates the inherent connection between area and perimeter in geometry. For a square, when the area is determined, the side length is uniquely determined, and therefore the perimeter is also uniquely determined. This method of deriving the perimeter from the area is frequently used in practical applications such as architectural design and land surveying. Therefore, the perimeter of this square is 24 cm.

[0113] Step S1 includes the following specific steps, namely S11~S12.

[0114] S11. Through the encoding process of a sparse autoencoder, the hidden layer representation of the large language model is projected onto a high-dimensional sparse atomic space, expressed as:

[0115] ;

[0116] In the formula, This is the sparse representation after encoding; This represents the hidden layer representation of a large language model. , For the set of real numbers, For sequence length, For hidden layer dimensions; For encoding the weight matrix, For atomic space dimension, , The expansion coefficient, used to balance expressive power and sparsity, is generally set to... ; To encode the bias vector, it is extended through a broadcast mechanism. Add the dimensions together; As the activation function, the JumpReLU activation function is introduced to control the activation threshold to achieve sparsity, improve sparsity, and ensure that only semantically significant atoms are activated.

[0117] S12. Reconstruct the original representation from the encoded sparse representation using a decoder, as shown in the expression:

[0118] ;

[0119] In the formula, for The result after reconstructing the original representation; To decode the weight matrix; This is the decoding bias vector.

[0120] The sparse autoencoder is optimized by minimizing the combination loss function, expressed as:

[0121] ;

[0122] In the formula, This is the combined loss; Represents the L2 norm. Represents the L1 norm; This is to reconstruct the error term, ensuring that the original information is preserved as much as possible during the encoding and decoding process; As a sparsity constraint term, the activation values ​​of the hidden layer are constrained. The sparsity forces the model to learn more concise core features; These are the sparsity weighting coefficients. Training was performed using the AdamW optimizer, with a learning rate set to [value missing]. Weight decay is set to The batch size was set to 128, the gradient accumulation steps were set to 4, and the convergence condition was set to the validation set reconstruction error being less than 5% and no improvement for 5 consecutive training epochs.

[0123] S2. Perform statistical analysis on comparative datasets with different inference depths to identify and classify extended thinking atoms that promote inference expansion and convergent thinking atoms that promote inference convergence.

[0124] It should be noted that the concept of "atom of thought" is proposed in an existing paper (Atom of Thoughts for Markov LLM Test-Time Scaling). Its core idea is that complex reasoning can be achieved through a series of lightweight "atomic problems," the execution of which depends only on themselves, freeing them from reliance on historical information. AoT constructs the reasoning process as a Markov process, continuously simplifying the problem through state transitions while maintaining equivalence to the original problem, ultimately solving the lightweight atomic problems to answer the original question.

[0125] This step identifies key atoms closely related to the expansion and convergence of the thought chain by comparing and analyzing atomic activation patterns at different inference depths, providing precise intervention targets for inference path regulation. (See...) Figure 4 For each atom, the activation statistics under different inference modes are calculated, and the activation differences of atoms are quantified using standardized effect sizes to distinguish between extended and convergent atoms. Step S2 includes the following specific steps, namely S21~S24.

[0126] S21. Construct a triplet comparison dataset ;in, Enter the query; For concise answers (length < 50 tokens); For detailed reasoning answers (length > 150 tokens); superscript Indicates a concise reasoning pattern, superscript Indicates a detailed reasoning pattern; This represents the number of samples.

[0127] In this embodiment, data construction follows the principle of semantic equivalence, and a semantic similarity model is used to... and Conduct the first round of semantic verification: Then, manual sampling is used to ensure an accuracy rate greater than 95%. and Maintain consistency in core conclusions, differing only in the degree of reasoning development, to avoid semantic differences in content interfering with the identification of atomic functions.

[0128] S22. For each thought atom Calculate its activation statistics under different inference modes; where the statistical parameters for the succinct mode are:

[0129] ;

[0130] ;

[0131] The statistical parameters in the detailed inference mode are:

[0132] ;

[0133] ;

[0134] In the formula, For the atom of thought Average activation strength in concise mode; For the atom of thought Average activation intensity in detailed reasoning mode; Representing the atom of thought In response to the problem Generate answer The average activation intensity during the process; Representing the atom of thought The standard deviation of activation intensity in concise reasoning mode; Representing the atom of thought The standard deviation of activation intensity in detailed reasoning mode.

[0135] S23. Based on the activation statistic calculated in step S22, the standardized effect size (Cohen's d) is used to quantify the activation differences of thought atoms, expressed as:

[0136] ;

[0137] In the formula, Indicates the first The effect size of a single thought atom.

[0138] S24. Classify thought atoms based on effect size, dividing them into extended thought atoms or convergent thought atoms; where, for extended thought atoms:

[0139] ;

[0140] For convergent thinking atoms:

[0141] ;

[0142] in, The set standard threshold is based on Cohen's d effect magnitude standard, and this embodiment uses... A value of 0.8 corresponds to a medium to large effect magnitude, ensuring that the identified atoms are statistically significant; For extended thinking atomic sets; It is a convergent thinking atom set.

[0143] S3. Obtain the text of the input question, assess the complexity of the input question based on the activation intensity of the extended thinking atoms identified from the input question, and map the complexity to the reasoning control strength.

[0144] Step S3 includes the following specific steps, namely S31~S32.

[0145] S31. Obtain the input problem. Define the complexity of the input problem based on the activation intensity of the extended thinking atoms. The expression is:

[0146] ;

[0147] In the formula, The input query is the current input question. In the atom of thought The activation value mentioned in step S11 above ; Weights are based on effect size; The sensitivity parameter controls the steepness of the sigmoid function; a larger value indicates a higher steepness. Values ​​make complexity distinctions more sensitive; smaller values... To make the distribution more even, a value can generally be set to... ,recommend ; The median complexity of the training set is obtained by using the training set. calculate:

[0148] ;

[0149] The sigmoid function is used to ensure To avoid extreme activation values ​​leading to abnormal results; To take the median.

[0150] S32. Map complexity to inference control strength, expressed as:

[0151] ;

[0152] In the formula, For reasoning control strength; To control the strength of the basic reasoning and ensure the basic reasoning quality for simple problems, the value range is generally set to be... This embodiment recommends ; To maximize control, provide sufficient depth of thought for complex problems, and avoid unnaturalness caused by over-control, the value range is generally set to [value range missing]. This embodiment recommends .

[0153] This mapping mechanism implements the following adaptive control logic:

[0154] Simple questions ( ): Approaching the basic control strength This allows the model to converge quickly, avoiding the waste of resources caused by overthinking;

[0155] Complex problems ): Approaching maximum control strength This promotes in-depth reasoning and prevents superficial answers from missing key information.

[0156] Medium-sized problems ( ): Control the intensity to transition smoothly in the middle range, and adaptively adjust the depth of thinking to match the needs of the problem.

[0157] S4. Based on the inference control strength and the thought atoms in the input question, generate an intervention vector. This step constructs a directional control vector based on the identified thought atoms and inference control strength to achieve precise intervention in the model's inference process, including the following specific steps, namely S41~S43.

[0158] S41. Extract semantic vectors from the model word embedding space and construct a bidirectional control vector, expressed as:

[0159] ;

[0160] ;

[0161] In the formula, For extended direction control vector; This is the convergence direction control vector; For the first The word embedding vector corresponding to each thought atom. The calculated effect strength Cohen's d value is used as the weighting coefficient.

[0162] In step S41, the bidirectional control vectors are further subjected to L2 normalization, as expressed by:

[0163] ;

[0164] ;

[0165] In the formula, This represents the L2 norm.

[0166] S42. Inference-based control strength Generate hybrid control vectors :

[0167] ;

[0168] This design implements the following control logic:

[0169] High control intensity ( ): Primarily employs extended vectors to facilitate deep reasoning;

[0170] Low control intensity ( ): Primarily employs convergent vectors to facilitate rapid conclusions;

[0171] Moderate control intensity: Balances expansion and convergence, and moderately guides the reasoning process.

[0172] S43. Generate intervention vectors :

[0173] ;

[0174] In the formula, This is the intervention intensity coefficient, used to control the degree of influence of the vector in the hidden space. A larger value... It will produce a stronger reasoning guidance effect, but may affect the generation quality; smaller The guiding effect is mild but may not be enough to change the reasoning pattern; it is generally assumed that... .

[0175] To ensure the numerical stability of the intervention vector, the intervention vector is also standardized, and the expression is:

[0176] ;

[0177] In the formula, To standardize the intensity and ensure the intervention vector is within a reasonable numerical range, the canonical norm of the model's hidden layer representation is generally set. .

[0178] S5. The intervention vector is applied to the intermediate hidden layer representation of the large language model to control the inference depth of the large language model and guide it to adaptively generate answer text according to the complexity of the input question. For example... Figure 5 As shown.

[0179] Step S5 includes the following specific steps, namely S51~S52.

[0180] S51. Select the intermediate layer of the large language model as the target intervention layer, the expression is:

[0181] ;

[0182] In the formula, The layer index represents the target intervention layer; This is the layer position coefficient, used to determine the relative location of the intervention; This represents the total number of layers in the large language model. This indicates rounding down. Shallow layers have unstable semantic representations, while deep layers have limited adjustment space. Generally, it's advisable to set... This corresponds to the 50%-70% mark of the total number of layers in the model. The principle behind selecting intermediate layers is that the model has already formed a relatively stable semantic representation at this point, but still has sufficient capacity for subsequent processing.

[0183] S52. Apply the intervention vector at the target intervention layer, expressed as:

[0184] ;

[0185] In the formula, For the original number Hidden layer representation, For the revised first Hidden layer representation; The intervention intensity is determined based on inference control intensity, and the expression is:

[0186] ;

[0187] In the formula, This is the intervention amplification factor, which converts the inference control strength into the hidden layer intervention strength. A larger one... It will produce a stronger intervention effect, but may affect the model's natural generation ability; smaller The intervention is mild but may not be very effective. In this embodiment, the intervention amplification factor is set to... .

[0188] To avoid representation space shift caused by excessive intervention, an adaptive residual connection is used for smooth transition, expressed as:

[0189] ;

[0190] In the formula, For the first The final intervention result represented by the hidden layer; The mixing coefficient is adaptively adjusted based on the intervention intensity.

[0191] ;

[0192] In the formula, min(·) selects the minimum value, and max(·) selects the maximum value.

[0193] This design ensures that the correction weights are appropriately increased when the intervention intensity is high, while maintaining the dominance of the original representation when the intervention is weak.

[0194] To maintain the numerical stability of the hidden layer representation and prevent gradient anomalies caused by intervention, layer normalization is performed on the final result:

[0195] ;

[0196] This operation ensures that the representation after intervention maintains the same numerical distribution characteristics as the original representation, thus avoiding affecting the normal processing of subsequent layers.

[0197] This embodiment presents a specific example: depth control of mathematical inference based on the Llama-3.1-8B model.

[0198] by Figure 2 The implementation process of this invention is illustrated using a specific mathematical reasoning task as an example. Suppose that two mathematical problems of different complexities are input: the simple problem "If the area of ​​a square is 36 square centimeters, what is its perimeter?" and the complex problem "Prove that for any positive integer n, 1+2+3+...+n = n(n+1) / 2".

[0199] First, the hidden layer representation of layer 15 of the Llama-3.1-8B model is decomposed using a pre-trained sparse autoencoder (SAE). Assuming the hidden layer dimension D=768, the SAE projects it into a sparse atom space of M=8192 dimensions. By analyzing a mathematical inference dataset containing 1000 short inference samples and 1000 long inference samples, 186 extended atoms and 94 convergent atoms are identified.

[0200] For the simple question "If the area of ​​a square is 36 square centimeters, what is its perimeter?", the activation of extended atoms in its atomic activation vector is relatively weak, resulting in a complexity C(x) = 0.28. The corresponding inference control strength λ(x) = 0.304 indicates a tendency towards convergence. An intervention vector is generated based on this inference control strength, and after standardization, it is applied to the 15th hidden layer representation. After intervention, the model produces a concise and direct inference process: "According to the formula for the area of ​​a square, the side length = ... =6 cm, circumference =4×6=24 cm.

[0201] For the complex problem "proving that for any positive integer n, 1+2+3+...+n = n(n+1) / 2", the extended atoms in the atomic activation vector are significantly activated, resulting in a complexity C(x) = 0.82 and a complexity C(x) = 0.61. The corresponding inference control strength λ(x) = 0.601 indicates a bias towards the extended mode. An intervention vector is generated based on this inference control strength and, after standardization, is applied to the 15th hidden layer representation. After intervention, the model performs a detailed inductive proof inference process, including basic step verification (the equation holds when n=1), inductive hypothesis establishment (assuming the equation holds when n=k), inductive step derivation (proving the equation also holds when n=k+1), and a conclusion summary.

[0202] In summary, the present invention has the following advantages:

[0203] First, a thought atom recognition technology based on a sparse autoencoder was constructed. By using the JumpReLU activation function, the high-dimensional hidden layer representation of a large language model is projected onto a higher-dimensional sparse atomic space. By utilizing a combination optimization strategy of L2 reconstruction loss and L1 sparsity constraints, an invertible mapping from dense representation to interpretable atomic representation is achieved, providing interpretable semantic units for precise manipulation of the reasoning process.

[0204] Secondly, an extended-convergent biatomic classification mechanism based on Cohen's d effect size was established. By constructing a triplet comparison dataset, the difference in atom activation at different inference depths was quantified using the standardized effect size Cohen's d, accurately identifying extended atoms that promote inference extension and convergent atoms that promote inference convergence, thus achieving precise regulation of the inference process.

[0205] Finally, an adaptive control algorithm based on problem complexity awareness was designed. A complexity evaluation function was constructed by quantifying the weighted activation intensity of extended thinking atoms, establishing a mapping mechanism from complexity to inference control intensity. Combined with control vector generation and hidden layer intervention techniques, this enabled automatic adjustment of the inference depth control intensity and precise guidance of the inference process.

[0206] Example 2

[0207] This embodiment provides a reasoning control system for a large language model thinking chain, applying the reasoning control method for a large language model thinking chain described in Embodiment 1; the reasoning control system includes: a sparse atom representation learning module, a thinking atom recognition and classification module, a complexity perception mechanism module, an intervention vector generation module, and a reasoning process intervention control module.

[0208] The sparse atom representation learning module is used to decompose the hidden layer representation of a large language model into a higher-dimensional sparse atom space using a sparse autoencoder, thereby constructing an invertible mapping from dense representation to sparse atom representation.

[0209] The Mind Atom Recognition and Classification Module is used to perform statistical analysis on comparative datasets with different inference depths, thereby identifying and classifying extended mind atoms that promote inference expansion and convergent mind atoms that promote inference convergence.

[0210] The complexity-aware mechanism module is used to acquire the text of the input question, evaluate the complexity of the input question based on the activation intensity of the extended thinking atoms identified from the input question, and map the complexity to the inference control strength.

[0211] The intervention vector generation module is used to generate an intervention vector based on the inference control strength and the thought atoms in the input question.

[0212] The reasoning process intervention control module is used to apply the intervention vector to the intermediate hidden layer representation of the large language model in order to control the reasoning depth of the large language model and guide the large language model to adaptively generate answer text according to the complexity of the input question.

[0213] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A reasoning control method of a large language model thought chain, characterized by, The method comprises the following steps: S1. decomposing the hidden layer representation of a large language model into a higher-dimensional sparse atom space using a sparse autoencoder to construct a reversible mapping from dense representation to sparse atom representation; S2. performing statistical analysis on the comparative data set of different reasoning depths to identify and classify the expansion-type thinking atoms that promote reasoning expansion and the convergence-type thinking atoms that promote reasoning convergence; S3. obtaining the text of the input question, evaluating the complexity of the input question according to the activation intensity of the expansion-type thinking atoms identified from the input question, and mapping the complexity to reasoning control intensity; S4. generating an intervention vector based on the reasoning control intensity and the thinking atoms in the input question; S5. applying the intervention vector to the intermediate hidden layer representation of the large language model to control the reasoning depth of the large language model and guide the large language model to adaptively generate answer text according to the complexity of the input question; Step S1 comprises the following specific steps: S11. projecting the hidden layer representation of the large language model to the high-dimensional sparse atom space through the encoding process of the sparse autoencoder, and the expression is: wherein, is the encoded sparse representation; is the large language model hidden layer representation, , is the set of real numbers, is the sequence length, is the hidden layer dimension; is the encoding weight matrix, is the atomic space dimension, , is the expansion coefficient for balancing expressiveness and sparsity; is the encoding bias vector, which is expanded to dimensions by a broadcasting mechanism and added; is the activation function; S12. reconstructing the original representation through the decoder after encoding the sparse representation, and the expression is: wherein is the result of reconstructing the original representation; is a decoded weight matrix; is a decoded bias vector.

2. The reasoning control method of a large language model thought chain according to claim 1, characterized in that, The sparse autoencoder is optimized by minimizing the combined loss function, and the expression is: wherein is the combined loss; denotes the L2 norm, denotes the L1 norm; is the reconstruction error term; is the sparsity constraint term; is the sparsity weight coefficient, ; The training adopts AdamW optimizer, the learning rate is set to , the weight decay is set to , the batch size is set to 128, the gradient accumulation step is set to 4, and the convergence condition is set to the reconstruction error of the validation set being lower than 5% and no improvement for 5 consecutive training periods.

3. The reasoning control method of a large language model thought chain according to claim 1, characterized in that, Step S2 comprises the following specific steps: S21. Constructing a Triplet Contrast Dataset ; wherein, is an input query; is a concise answer; is a detailed reasoning answer; the superscript denotes a concise reasoning mode, the superscript denotes a detailed reasoning mode; is a number of samples; S22. For each thought atom , calculate its activation statistics under different reasoning modes; wherein, the statistical parameters under the concise mode are: The statistical parameters in the detailed reasoning mode are: wherein is the thought atom average activation strength in concise mode; is the thought atom average activation strength in detailed reasoning mode; is the thought atom activation strength in response to a question generating an answer average activation strength in the process; is the thought atom standard deviation of activation strength in concise reasoning mode; is the thought atom standard deviation of activation strength in detailed reasoning mode; S23. Based on the activation statistics calculated in step S22, the activation difference of the thinking atoms is quantified using a standardized effect size, and the expression is: wherein represents the effect quantity of the th thought atom; S24. Based on the size of the effect size, the thinking atoms are classified into expansion-type thinking atoms or convergence-type thinking atoms; wherein, for the expansion-type thinking atoms: For the convergence-type thinking atoms: wherein, is a set of standard thresholds; is a set of extended thinking atoms; is a set of convergent thinking atoms.

4. The reasoning control method of a large language model thought chain according to claim 3, characterized in that, Step S3 comprises the following specific steps: S31. Obtain the input question, define the complexity of the input question based on the activation intensity of the expansion-type thinking atoms, and the expression is: wherein, is the input query, i.e. the current input question is the activation value on the thought atom ; is the weight based on the effect size; is the sensitivity parameter, controlling the steepness of the sigmoid function; is the training set complexity median, computed over the training set ; ; the sigmoid function is used to ensure ; is the median taken; S32. Map the complexity to the reasoning control intensity, and the expression is: In the formula, is the reasoning control strength; is the base control strength, and the value range is ; is the maximum control strength, and the value range is .

5. The reasoning control method of a large language model thought chain according to claim 4, characterized in that, Step S4 comprises the following specific steps: S41. Extract semantic vectors from the model word embedding space to construct a bidirectional control vector, and the expression is: In the formula, For extended direction control vector; This is the convergence direction control vector; For the first The word embedding vector corresponding to each thought atom; S42. Control intensity based on inference Generating a hybrid control vector : S43. Generating an intervention vector : In the formula, is an intervention strength coefficient for controlling the degree of influence of the control vector in the hidden layer space.

6. The reasoning control method of a large language model thought chain according to claim 5, characterized in that, In step S41, the bidirectional control vector is also subjected to L2 normalization processing, and the expression is: wherein denotes the L2 norm; In step S43, the intervention vector is also standardized, and the expression is: In the formula, is the normalized intensity.

7. The reasoning control method of a large language model thought chain according to claim 5, characterized in that, Step S5 comprises the following specific steps: S51. Select the middle layer of the large language model as the target intervention layer, and the expression is: In the formula, represents the layer index of the target intervention layer; is a layer position coefficient for determining the relative position of the intervention; is the total number of layers of the large language model; represents the floor function; S52. Apply the intervention vector to the target intervention layer, and the expression is: In the formula, is the original first layer hidden layer representation, is the modified first layer hidden layer representation; is the intervention strength, determined based on the inference control strength, and the expression is: In the formula, To intervene the amplification coefficient, the inference control strength is converted into the hidden layer intervention strength.

8. The reasoning control method of a large language model thought chain according to claim 7, characterized in that, The intervention amplification factor is set to ; wherein a smooth transition is performed using an adaptive residual connection, expressed as: In the formula, is the first layer hidden layer representation of the final intervention result; is a mixing coefficient, min(·) is the minimum value selection, and max(·) is the maximum value selection. After step S52, the final intervention result is also subjected to layer normalization processing. layer normalization processing. 9.A reasoning control system of a thinking chain of a large language model, characterized by, A large language model thinking chain reasoning control method as claimed in any one of claims 1 to 8; the reasoning control system comprises: a sparse atom representation learning module for decomposing the hidden layer representation of a large language model into a higher-dimensional sparse atom space using a sparse autoencoder to construct a reversible mapping from dense representation to sparse atom representation; a thinking atom identification and classification module for performing statistical analysis on the comparative data set of different reasoning depths to identify and classify the expansion-type thinking atoms that promote reasoning expansion and the convergence-type thinking atoms that promote reasoning convergence; The complexity perception mechanism module is configured to acquire text of an input question, evaluate complexity of the input question according to activation intensity of an extended thinking atom identified from the input question, and map the complexity to reasoning control intensity; The intervention vector generation module is configured to generate an intervention vector based on the reasoning control intensity and the thinking atom in the input question; The reasoning process intervention control module is configured to apply the intervention vector to an intermediate hidden layer representation of the large language model to control a reasoning depth of the large language model, and guide the large language model to adaptively generate answer text according to the complexity of the input question.

Images