Decoupling method and device for slow-thinking and quick-response areas in large language model

Abstract

The invention discloses a method and a device for decoupling a slow-thinking area and a quick-response area in a large language model. The method comprises the following steps: constructing a slow thinking prompt and a fast response prompt, respectively inputting the slow thinking prompt and the fast response prompt into a large language model, and recording a neuron activation value of each linear layer of the large language model; calculating an activation value difference vector based on the neuron activation values corresponding to the slow thinking prompt and the fast response prompt, and obtaining a prompt difference vector based on the activation value difference vector; iteratively processing a plurality of reasoning problems, obtaining positive value position intersection of prompt difference vectors in all reasoning problems, and identifying a key neuron set related to slow thinking; and constructing a gradient mask matrix based on the position information of the key neurons in the key neuron set, and performing fine tuning on the key neurons by adopting a mask mode based on the gradient mask matrix to obtain an optimized large language model. The technical problem that an existing model is poor in reasoning ability is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a method and device for decoupling slow thinking and fast response regions in a large language model. BACKGROUND

[0002] With the wide application of large language models (LLMs) in natural language processing tasks, they have shown excellent performance in translation, summarization, question answering and other routine tasks. However, in high-order cognitive tasks such as mathematical reasoning, complex planning and systematic logical reasoning, existing large language models still show certain limitations. The reason for this deficiency in reasoning ability is that there is a lack of effective activation of the "slow thinking (System-2)" mechanism within the model.

[0003] In recent years, existing technologies have attempted to guide the model to generate a step-by-step thinking process through process supervision models, reinforcement learning and chain-of-thought prompts (CoT), etc., in order to improve reasoning performance. Although these methods have improved the accuracy of the model's answers to some extent, they are still mainly focused on optimizing the output results, and have not yet revealed the working principle of the internal reasoning mechanism of the model and the distribution of key parameters.

[0004] Existing parameter-efficient fine-tuning (PEFT) methods such as LoRA, BitFit and Half Fine-Tuning, although they can save computing power while improving model performance, have a coarse operation granularity and usually only fine-tune specific layers or modules, making it difficult to accurately locate and optimize the key neurons that truly drive the model's reasoning ability. In addition, most methods rely on backpropagation to calculate gradient information to select parameters to be fine-tuned, which not only has high computational cost, but also lacks a clear explanation of the cognitive function of the selected regions.

[0005] Therefore, there is an urgent need for a technology that can effectively identify and utilize the neuron regions in large language models that are closely related to the slow thinking process, in order to achieve a more explainable and more efficient model reasoning ability enhancement path.

[0006] To address the above problems, no effective solutions have been proposed so far. SUMMARY

[0007] The embodiments of the present application provide a method and device for decoupling slow thinking and fast response regions in a large language model, to at least solve the technical problem of poor model reasoning ability.

[0008] According to one aspect of an embodiment of the present invention, a method for decoupling slow thinking and fast response areas in a large language model is provided, comprising: constructing slow thinking prompts and fast response prompts, inputting the slow thinking prompts and the fast response prompts into the large language model respectively, and recording the neuron activation values ​​of each linear layer of the large language model; calculating an activation value difference vector based on the neuron activation values ​​corresponding to the slow thinking prompts and the fast response prompts, respectively, and obtaining a prompt difference vector based on the activation value difference vector; iteratively processing multiple reasoning problems, obtaining the intersection of the positive value positions of the prompt difference vectors in all reasoning problems, and identifying a key neuron set related to slow thinking; validating the key neuron set, and analyzing the change in the reasoning performance of the large language model through different degrees of unstructured pruning and activation strength suppression; constructing a gradient mask matrix based on the position information of key neurons in the key neuron set, and fine-tuning the key neurons using a masking method based on the gradient mask matrix to obtain an optimized large language model.

[0009] According to another aspect of an embodiment of the present invention, a method for decoupling slow thinking and fast response areas in a large language model is provided, including: a construction module, configured to construct slow thinking prompts and fast response prompts, input the slow thinking prompts and the fast response prompts into the large language model respectively, and record the neuron activation values ​​of each linear layer of the large language model; a calculation module, configured to calculate an activation value difference vector based on the neuron activation values ​​corresponding to the slow thinking prompts and the fast response prompts, and obtain a prompt difference vector based on the activation value difference vector; an identification module, configured to iteratively process multiple reasoning problems, obtain the positive position intersection of the prompt difference vectors in all reasoning problems, and identify a key neuron set related to slow thinking; a verification module, configured to verify the key neuron set and analyze the change in reasoning performance of the large language model through different degrees of unstructured pruning and activation strength suppression; a fine-tuning module, configured to construct a gradient mask matrix based on the position information of key neurons in the key neuron set, and fine-tune the key neurons using a masking method based on the gradient mask matrix to obtain an optimized large language model.

[0010] In the embodiment of the present application, a slow thinking prompt and a fast response prompt are constructed, the slow thinking prompt and the fast response prompt are input into a large language model respectively, and neuron activation values of each linear layer of the large language model are recorded; an activation value difference vector is calculated based on the neuron activation values corresponding to the slow thinking prompt and the fast response prompt respectively, and a prompt difference vector is obtained based on the activation value difference vector; a plurality of reasoning problems are iteratively processed, a positive value position intersection of the prompt difference vectors in all reasoning problems is obtained, and a key neuron set related to slow thinking is identified; the key neuron set is verified, reasoning performance changes of the large language model are analyzed through different degrees of unstructured pruning and activation intensity suppression; a gradient mask matrix is constructed based on position information of key neurons in the key neuron set, the key neurons are fine-tuned in a mask mode based on the gradient mask matrix, and an optimized large language model is obtained. Through the above scheme, the technical problem of poor reasoning ability of the existing model is solved. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0012] Figure 1 is a flowchart of a method for decoupling slow thinking and fast response regions in a large language model according to an embodiment of the present application;

[0013] Figure 2 is a flowchart of another method for decoupling slow thinking and fast response regions in a large language model according to an embodiment of the present application;

[0014] Figure 3 is a flowchart of an optional prompt difference vector acquisition method according to an embodiment of the present application;

[0015] Figure 4 is a flowchart of a simulation experiment method for decoupling slow thinking and fast response regions in a large language model according to an embodiment of the present application

[0016] Figure 5 is a graph showing the decline trend of the number of neurons at different levels in the process of neuron exploration according to an embodiment of the present application, wherein (a), (b) and (c) respectively show the decreasing trend of the number of neurons in the shallow, middle and deep feedforward network (FFN) of the model;

[0017] Figure 6 is an example of large language model output before pruning vs. random neuron pruning vs. slow thinking neuron pruning according to an embodiment of the present application;

[0018] Figure 7 is a structural diagram of a slow thinking and fast response area decoupling device in a large language model according to an embodiment of the present application;

[0019] Figure 8 A structural schematic diagram of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0020] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0022] According to an embodiment of the present application, a method embodiment of a slow thinking and fast response area decoupling method in a large language model is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0023] Figure 1 is a slow thinking and fast response area decoupling method in a large language model according to an embodiment of the present application, as Figure 1 shown, the method comprises the following steps:

[0024] Step S102, constructing slow thinking prompts and fast response prompts, inputting the slow thinking prompts and the fast response prompts into a large language model, and recording the neuron activation values of each linear layer of the large language model.

[0025] Neurons in large language models have specific functions. For example, skill neurons identify by comparing the activation value of a neuron to the average activation value of the layer, and selecting neurons that exceed the baseline value. Other techniques use sparse probing techniques to divide neurons into different categories, including monosemantic neurons (responding to a single feature), polysemantic neurons (activated by multiple features), context neurons (sensitive to high-level sequence features such as language or code), and compound word neurons (representing compound words through semantic combination). In addition, repetition neurons have gradually increasing activation values when processing repeated sequences, and can treat "repetition" as a task that can be artificially reinforced. Through linear regression probing methods, spatial neurons and temporal neurons that encode entity positions and temporal dynamics are also identified, enabling the model to have complex spatio-temporal reasoning and sequence representation capabilities, significantly improving its ability to process spatio-temporal information.

[0026] Step S104, based on the neuron activation values corresponding to the slow thinking prompt and the fast response prompt respectively, calculate the activation value difference vector, and obtain the prompt difference vector based on the activation value difference vector.

[0027] For example, difference operations are performed on the activation values of each layer of neurons corresponding to the slow thinking prompt and the fast response prompt respectively, to obtain the activation value difference vector of each layer.

[0028] Step S106, iteratively process multiple reasoning problems, obtain the positive value position intersection of the prompt difference vectors in all reasoning problems, and identify the key neuron set related to slow thinking.

[0029] Mathematical reasoning ability is a key indicator of the intelligence level of large language models (LLMs). The "model self-consistency" method has been proven to significantly improve reasoning performance. This method samples multiple different Chain-of-Thought (CoT) paths from the model's decoder, and then selects the most consistent answer as the final output. Experiments have shown that self-consistency significantly improves mathematical reasoning performance on multiple public benchmarks.

[0030] In addition, converting natural language solutions to Python programs and verifying them can effectively eliminate incorrect reasoning paths, ensuring that only correct solutions that can be verified are retained, thereby reducing the risk of the model consistently generating incorrect answers.

[0031] From the perspective of data sets, it is also very important to build high-quality mathematical data sets. The quality of the data set not only includes diversity and richness, but also includes size. For example, as the size of the generated data increases, the model performance shows a logarithmic growth trend. On this basis, key knowledge points and their corresponding question and answer pairs are extracted from real data, classified by topic (such as algebra, geometry, number theory), and a topic library is constructed. By sampling knowledge points from this library to generate new questions, the KP-math data set is constructed, containing more than one million question and answer pairs. By merging with existing data sets (such as MATH, MMIQC, MathInstruct-CoT), an extended version of the KPMath-Plus data set is formed. Fine-tuning the language model on KP-math significantly improves the mathematical reasoning ability, and the zero-shot accuracy is significantly improved compared to the baseline model.

[0032] Similarly, MetaMathQA introduces data augmentation strategies (such as answer expansion, question restatement, reverse reasoning), which enhance data diversity and cover forward and reverse reasoning paths with 395,000 questions, further improving the coverage and depth of the training set.

[0033] Arithmetic ability is considered a core component of mathematical reasoning, and specific arithmetic neurons exist in the model; other work positions the weight parameters strongly related to mathematical tasks at a higher level, and ablation experiments show that these parameters have a decisive influence on mathematical ability, while having limited impact on other tasks. In addition, steering vectors can effectively activate the model's thinking chain process, and have better performance than traditional prompt engineering in reasoning tasks.

[0034] In this embodiment, for each reasoning sample, the index of all positively activated neurons is extracted; the intersection of all positively activated neuron indices between multiple reasoning samples is taken to obtain a set of neurons that are always highly activated under the slow thinking prompt, which is used as a set of key neurons related to slow thinking.

[0035] Step S108, verifying the key neuron set by different degrees of unstructured pruning and activation intensity suppression, and analyzing the reasoning performance changes of the large language model.

[0036] The key neuron set is pruned by 50%, 75%, and 100%, and the activation intensity is suppressed, and the reasoning coherence, arithmetic accuracy, and structural integrity of the large language model are evaluated in multi-step reasoning tasks.

[0037] Step S110, based on the position information of the key neurons in the key neuron set, a gradient mask matrix is constructed, and the key neurons are fine-tuned in a masked manner based on the gradient mask matrix, to obtain an optimized large language model.

[0038] Delta tuning is a variant of parameter-efficient fine-tuning (PEFT) that incorporates a fine-tuning strategy based on a specific task specification. This approach uses heuristics to selectively adjust some weight parameters, thereby improving model performance in a specific domain while preserving most of the original parameters. In some cases, this approach can even surpass full parameter fine-tuning (FFT). For example, gradient mask tuning (GMT) performs preliminary fine-tuning on a small batch of training data. It then selects target parameters for fine-tuning based on the locations of parameters in the gradient matrix that exceed a preset threshold k. The BitFit method, on the other hand, fine-tunes only the biases, outperforming FFT on small datasets. The effect is particularly pronounced in the query layer and intermediate multilayer perceptron (MLP) layers, where adjusting only the biases can mimic the effect of full bias tuning. Another approach, called partial fine-tuning, ranks parameters based on gradient norm and activation value, fine-tuning the top 50% and freezing the rest. Even randomly selecting parameters for fine-tuning has been shown to be effective, and experiments show that fine-tuning only 1% of the parameters can achieve competitive performance.

[0039] However, existing methods often require costly backpropagation to obtain gradient information, which imposes a significant burden in practical deployments. While random selection methods are more convenient, they lack a theoretical foundation, resulting in poor interpretability and unstable performance. To address these issues, this application proposes a novel parameter selection strategy that combines the advantages of existing methods while overcoming their limitations, providing a computationally efficient, operationally simple, and biologically inspired solution.

[0040] Specifically, in this embodiment, first, a gradient mask matrix is ​​constructed. Based on the position information of the key neurons in the key neuron set, the position corresponding to the position information in the weight matrix of the key neurons in the key neuron set is set to 1, and the remaining positions are set to 0 to construct the gradient mask matrix. Then, the key neurons are fine-tuned. A sparse update method is used to retain only the gradient updates with a mask position of 1 in the gradient mask matrix, and the rest are frozen. The updated values ​​are written to the corresponding weight matrix positions through dynamic mapping to fine-tune the key neurons.

[0041] The method of NeuroCoT proposed in the present application can isolate the inference and analysis regions in the model by only forward propagation. Unstructured pruning of this region can significantly weaken the model's ability to generate chain-of-thought; and fine-tuning of this region can obtain better performance compared to other parameter efficient fine-tuning (PEFT) methods. The present application first identifies neurons representing "slow thinking", and the neurons in this region play a significantly higher role in the model generating chain-of-thought than other regions, so that there is a special inference region similar to the "slow thinking system" of the human brain in large language models.

[0042] Figure 2 Another method for decoupling slow thinking and fast response regions in a large language model according to an embodiment of the present application is shown in Figure 2 The method comprises the following steps:

[0043] Step S202, identify slow thinking neurons.

[0044] The present method uses prompt engineering to guide the model to generate different styles of answers by appending two different prompts after complex inference questions: (a) "Please think step by step and answer the following question" (referred to as slow thinking prompt); (b) "Please answer the following question immediately without detailed explanation" (referred to as fast thinking prompt). These prompts will be input into the large language model (LLM) for processing.

[0045] The hook function of PyTorch is used to obtain the activation value of each linear layer and store it in a dictionary. Then, the vector difference between the neuron activation values corresponding to the two prompts is calculated to obtain the "prompt difference vector".

[0046] When the value of some position in the vector is greater than 0, it means that the higher activation value at this position helps the model to generate a coherent reasoning process. However, analyzing a single question alone is not enough. In order to extract the activation pattern that is unique to slow thinking or fast thinking mode, the present embodiment designs an iterative intersection algorithm based on 464 inference questions:

[0047] 1. Perform the calculation steps of the "prompt difference vector" for the first question;

[0048] 2. For subsequent questions, calculate the current prompt difference vector and perform intersection with the positive value positions in the last iteration result.

[0049] The pseudo code of the method is shown in Algorithm 1. After processing all questions, the remaining positive value positions in the prompt difference vector show a significant and persistent correlation with the "chain-of-thought" (CoT) generation. These residual neural dimensions can be considered as key activation features that support systematic reasoning. Figure 3 A schematic diagram illustrating a prompt difference vector acquisition process is shown.

[0050] Algorithm 1: Slow thinking neuron identification

[0051] Input: a pre-trained model

[0052]

[0053] Step S204, verification is performed.

[0054] In order to verify the accuracy of the selected slow thinking neurons, unstructured pruning is performed on them, and a hierarchical neuron inhibition strategy is combined. This step effectively verifies the role of these neurons in the reasoning process and lays the foundation for subsequent fine-tuning.

[0055] By comparing the model output with different pruning intensities (50%, 75%, 100%), the performance in terms of reasoning step integrity, arithmetic accuracy and structural coherence is investigated. The results show that multi-step reasoning tasks (such as the Betty saving problem) are more sensitive to pruning, while single-step problems (such as the Natalia editing problem) are less affected.

[0056] Table 1 shows that the hierarchical neuron inhibition strategy proposed in this application can controllably weaken the structured reasoning ability under different pruning degrees. When the weakening is 50%, the reasoning ability of the model remains basically unchanged; when the weakening is 75%, the model makes obvious errors in multi-step reasoning, such as missing the initial amount of money held by Betty; in single-step problems, the reasoning process becomes too simple and lacks a final conclusion. When the weakening is 100%, the model completely skips the intermediate reasoning steps and directly outputs an error answer without thinking.

[0057] This ablation experiment shows that slow thinking neurons are positively correlated with the thinking chain process of the model, and operations on these neurons will gradually weaken the reasoning ability of the model.

[0058] In addition, special attention should be paid to the location of super activations and superweights in the model. In large language models, setting these key values to zero may cause a significant decline in model performance, and even output meaningless content, such as repeated "the" or periods. Therefore, these values should be preserved during processing to ensure that the basic language ability of the model is not damaged, so that the evaluation results have practical significance.

[0059]

[0060] Table 1

[0061] Step S206, neuron level fine-tuning.

[0062] 1) Measure the importance of neurons in large models.

[0063] Modern large language models usually adopt the Transformer architecture, the most common of which is a structure containing only decoders. The decoder is composed of multiple identical decoder layers, each of which contains a self-attention module and a feed-forward neural network module. These modules are composed of multiple linear layers, which contain distributed neurons inside. This embodiment focuses on the neurons in these linear layers.

[0064] Take the feed-forward layer as an example, its form is expressed as follows:

[0065] FFN(x) = f(xW k +b1)W v +b2 (1) where Wkand Wvare trainable weight matrices, x is the residual stream from the previous module, f is the activation function, and b1and b2are bias parameters.

[0066] Definition:

[0067]

[0068] where dmrepresents the hidden layer dimension after a linear transformation. Based on the principle of block matrix multiplication, the i-th element of vector a i can be regarded as the "importance score" of the i-th column v i in the weight matrix Wv. The larger a i , the greater the contribution of the column to the next layer of hidden states, and thus the greater the impact on the final output of the model; if a i is 0, it means that the column has no effect on the output. Therefore, each column of the weight matrix is defined as a neuron, and the importance of the neuron is measured by the activation value of the hidden layer.

[0069] 2) Determine the gradient masking.

[0070] For selected slow thinking neurons, record their specific positions in the weight matrix, denoted as W(l, n), where W represents a specific weight matrix, l represents the model layer number, and n represents the column number in the matrix.

[0071] Based on this position information, a mask matrix composed of 0 and 1 is constructed: 1 indicates that the parameter at this position will be updated during fine-tuning, and 0 keeps the original value unchanged. The gradient masking technique ensures that only selected positions are updated during fine-tuning, and other parameters remain frozen, thereby achieving fine-grained control at the neuron level.

[0072] This application fine-tuned the model using the WebInstruct-CFT dataset to improve its reasoning capabilities, focusing specifically on the model's "slow thinking" areas. This dataset, distilled from GPT-4o output, covers mathematics, physics, and chemistry, offering high-quality data and reliable supervisory signals. Experimental results demonstrate that fine-tuning with WebInstruct-CFT significantly improves model performance on multiple mathematical benchmarks, further validating the effectiveness of critical training strategies in enhancing the reasoning capabilities of language models.

[0073] Figure 4 This is another simulation experimental method for decoupling the slow thinking and fast response areas in a large language model according to an embodiment of the present invention. Figure 4 As shown, the method includes the following steps:

[0074] Step S402: Select a model.

[0075] In this example, the Llama3.2-3B-Inst and Qwen2.5-7B models were selected for performance evaluation during the fine-tuning phase. The Llama3.2-3B-Inst was also used in ablation experiments. The Llama3.2-3B-Inst was chosen because it has excellent instruction-following capabilities, facilitating the extraction of "cue difference vectors" using structured cues, enabling deeper analysis of the model's internal decision-making mechanisms.

[0076] To verify the effectiveness of the NeuroCoT method, experiments were conducted using models with 3B and 7B parameters, aiming to evaluate whether NeuroCoT can maintain consistent results under different scales, architectures, and pre-training strategies.

[0077] Step S404: Select a data set.

[0078] The core goal of this application is to identify neurons and areas related to model thinking or reasoning ability. Therefore, the selected dataset must be able to fully stimulate the model's potential thinking ability. To this end, the following five datasets were selected for selecting "slow thinking neurons": GSM8K, MMLU, ARC-Easy, ARC-Hard, and TruthfulQA. Specifically, 100 mathematical reasoning questions were randomly selected from GSM8K, 2 questions were selected from each subject of MMLU, and 100 questions were selected from ARC-Easy, ARC-Hard, and TruthfulQA, totaling 464 samples. This selection strategy takes into account the diversity of reasoning questions and data scale control, thereby improving the efficiency of neuron selection.

[0079] These datasets cover a variety of reasoning tasks, such as mathematical reasoning, scientific reasoning, factual judgment, and multi-domain knowledge. Combined with the Chain-of-Thought prompt technology, they effectively stimulate the model's deep thinking ability and provide support for analyzing its internal mechanisms.

[0080] Step S406: fine-tune the settings and compare the baseline.

[0081] 1) Training hyperparameters

[0082] For fine-tuning, the mini-batch size was set to 2, and the number of gradient accumulation steps was set to 128 to offset the inherent noise in mini-batch training. To maintain training stability, the learning rate was set to 5e-7, and the warm-up ratio was set to 0.1. Since all samples in the dataset were less than 2048 tokens long, the maximum sequence length was set to 2048.

[0083] In addition, the two models are trained using WebInstruct-CFT datasets of different scales: Llama3.2-3B-Inst is fine-tuned using the WebInstruct-4k subset, and Qwen2.5-7B is fine-tuned using the larger WebInstruct-50k subset.

[0084] 2) Baseline Method

[0085] In order to rigorously evaluate the effectiveness of the method, this application was compared with the following mainstream baseline methods: Pre-trained (pre-trained model): keep the model in its original state and do not perform any fine-tuning for specific tasks as a performance reference; FFT (full parameter fine-tuning): update all model parameters during training to make it fully adapt to the target domain task; LoRA (low rank adaptation): fine-tune by learning a low-rank correction matrix, keeping most parameters unchanged, reducing the number of training parameters and computational cost; HFT (half-quantity fine-tuning): only fine-tune the parameters of the last 10 layers of the model, freezing the front layers to retain the original knowledge.

[0086] To comprehensively evaluate the model's reasoning capabilities, experiments were conducted on multiple representative benchmark datasets. For mathematical reasoning, the model was fine-tuned and evaluated on four authoritative benchmarks: Math-500, GSM8K, Minerva-Math, and AMC23. These datasets cover algebra, arithmetic, geometry, and competition-related problems, with varying levels of difficulty and multi-step reasoning. They effectively assess the model's capabilities and generalization in areas such as symbolic operations and quantitative analysis.

[0087] In addition to mathematical reasoning tasks, the model was also evaluated on the MMLU (Massive Multi-Task Language Understanding) benchmark to ensure that the NeuroCoT approach does not compromise its general domain performance. Since NeuroCoT only fine-tunes a sparse subset of neurons, it is necessary to verify whether this targeted adjustment preserves the overall model capability.

[0088] The MMLU covers 57 disciplines across the humanities, social sciences, natural sciences, and engineering, and is a key criterion for assessing general knowledge and reasoning. Its inclusion in the test will verify whether NeuroCoT improves mathematical reasoning while preserving generalized abilities.

[0089] During testing, a greedy decoding strategy was used to generate text. This strategy selects the token with the highest probability at each step to ensure the determinism and consistency of the generated results. Given the complexity of mathematical reasoning problems, the maximum generation length was set to 1024 tokens to ensure that the model has sufficient space to output a complete reasoning process. Furthermore, this example uses a zero-shot testing approach, directly testing the model without providing examples, to further verify its generalization and reasoning performance.

[0090] Step S408, perform analysis.

[0091] The experiment recorded the dynamic changes of neuronal detection. Figure 5 The figure shows the decreasing trend of the number of neurons in the upper and lower parts of the feed-forward network (FFN) in different layers of the LLaMA3.2-3B-Inst model.

[0092] Experimental results show that the number of neurons gradually decreases and eventually stabilizes. In the middle and later layers of the model, the number of neurons retained after algorithm iteration does not exceed 100. In addition, the convergence speed of different layers is different: the first 10 layers converge significantly slower than the subsequent 18 layers. This difference may be due to the fact that the shallow layers mainly process semantic information, so more steps are required to filter out areas related to pure reasoning; while the deep layers (such as layers 19 to 24) focus more on high-level logical reasoning or task-specific feature extraction (Jin et al., 2024), so the neuron detection process converges faster.

[0093] This phenomenon suggests that the deeper layers of LLMs may structure the learned knowledge through sparser but more targeted activation patterns.

[0094] Step S410, experimental results.

[0095] Fine-tuning the neurons selected by NeuroCoT falls into the category of parameter-efficient fine-tuning (PEFT). Therefore, two mainstream PEFT methods are additionally selected as comparative baselines: LoRA and HFT. The former is the most common PEFT method, and the latter utilizes the key role of the post-model layers in downstream tasks as a contrastive term.

[0096] In addition, based on the assumption that large models can be divided into slow thinking regions and fast response regions, traditional full-parameter fine-tuning is equivalent to adjusting both at the same time. Therefore, it is used as a benchmark to evaluate whether fine-tuning only the slow thinking region can achieve more cost-effective generalization and performance improvement.

[0097] 1) Mathematical reasoning results.

[0098] Table 2 shows the performance comparison of the five methods on mathematical tasks. In the Qwen2.5-7B model trained using the WebInstruct-50k dataset, NeuroCoT performs well on all four evaluation datasets, only slightly inferior to full-parameter fine-tuning (FFT). Notably, NeuroCoT only uses about 4% of the trainable parameters, while the generalization ability of other methods is unstable. For example, LoRA is slightly better than NeuroCoT on the Minerva-Math dataset, but performs poorly on Math-500, only about 2% better than the pre-trained model. At the same time, HFT has strong generalization ability on Math-500, but is inferior to NeuroCoT on GSM8K.

[0099] It is worth emphasizing that NeuroCoT performs better on small-scale datasets. In the Llama3.2-3B-Inst model, only 4000 samples in the WebInstruct-CFT dataset are used for training. Under such limited training conditions, full-parameter fine-tuning (FFT) exhibits significant overfitting on multiple datasets, with poor performance, possibly due to the model learning noise in the training data. NeuroCoT only fine-tunes about 3% of the parameters of FFT, effectively avoiding overfitting.

[0100] In addition, PEFT methods such as LoRA and HFT also limit parameter updates to improve generalization. As shown in Table 2, NeuroCoT performs similarly to FFT, but exhibits more stable generalization on multiple datasets.

[0101]

[0102] Table 2

[0103] 2) General domain ability.

[0104] This application further evaluated the model's performance on general language tasks to determine whether the NeuroCoT approach would affect overall abilities beyond mathematical reasoning. As shown in Table 3, the original pre-trained model and the model fine-tuned with NeuroCoT achieved almost identical test scores on the MMLU dataset, with the latter even achieving a slight improvement.

[0105]

[0106] Table 3

[0107] This result can be attributed to the fine-tuning strategy focusing solely on the "slow thinking" component responsible for generating chained thought processes, while leaving the model's fast response components unchanged. Since the MMLU dataset primarily requires the model to select an answer from four options (A, B, C, D), focusing on fast response capabilities, this test effectively verifies whether NeuroCoT affects the model's basic response capabilities. The results demonstrate that NeuroCoT does not weaken the model's general functionality or affect its performance on fast tasks.

[0108] 3) Ablation experiment.

[0109] In order to fully verify the advantages of neurons corresponding to the "slow thinking area" in large-scale language models compared with other areas, this application designed and implemented a series of systematic ablation experiments.

[0110] First, we conducted a pruning ablation experiment: one group pruned out slow-thinking neurons, while the other pruned out an equal number of random neurons. We then compared the outputs of the models before and after pruning. Figure 6 The performance of the model on a math problem in the GSM8K dataset is demonstrated. The results show that after pruning the slow thinking area, the model only outputs the final answer, omitting the intermediate reasoning steps; and after pruning the random neurons, the model can still generate detailed reasoning, similar to that before pruning. More importantly, pruning the slow thinking area significantly reduces the answer accuracy, highlighting the key role of this area in supporting the reasoning process. The second ablation experiment fine-tunes the slow thinking area and the random area separately to observe whether fine-tuning only specific areas can also improve model performance. Table 4 shows the performance improvement results obtained by fine-tuning the slow thinking area and fine-tuning the random area under the same training conditions. It can be seen that fine-tuning the random area not only fails to bring significant performance improvement, but also reduces the generalization ability; while fine-tuning the slow thinking area shows a clear advantage.

[0111]

[0112] Table 4

[0113] This further verifies the key value of slow thinking regions in complex reasoning tasks and shows that the targeted fine-tuning strategy based on region selection is more effective than the global strategy in improving the reasoning ability of the model.

[0114] The present application proposes a NeuroCoT method, which is a simple and direct framework for identifying "slow thinking regions" in large language models. Through pruning and inhibition experiments on neurons in these regions, the core role of these regions in reasoning performance is verified. Experiments show that weakening slow thinking neurons will significantly impair the reasoning ability of the model compared to other regions. The present application further fine-tunes neurons at the neuron level. By focusing on the "slow thinking region (STR)", which is a key subset of neurons supporting complex reasoning tasks, fine-grained parameter adjustment is performed. The results show that the performance improvement brought by fine-tuning the STR region is significantly better than that of random regions, and the new knowledge learned has stronger generalization ability on out-of-distribution (OOD) data. In addition, NeuroCoT performs particularly well when dealing with small-scale but high-quality datasets. This makes it particularly suitable for scenarios where data is limited but contains rich reasoning information, and for tasks that seek lightweight training. In this context, the fine-tuning achieved by NeuroCoT can be considered as a "targeted enhancement" or even "reconstruction" of the model's thinking chain, thereby achieving more efficient adaptation and stronger generalization ability while keeping the number of parameters controllable.

[0115] The present application also provides a decoupling device for slow thinking and fast response regions in a large language model, as shown in Figure 7 The device includes a construction module 72 configured to construct a slow thinking prompt and a fast response prompt, input the slow thinking prompt and the fast response prompt into a large language model, and record the neuron activation values of each linear layer of the large language model; a calculation module 74 configured to calculate an activation value difference vector based on the neuron activation values corresponding to the slow thinking prompt and the fast response prompt, and obtain a prompt difference vector based on the activation value difference vector; an identification module 76 configured to iteratively process multiple reasoning problems, obtain the intersection of positive value positions of the prompt difference vectors in all reasoning problems, and identify a key neuron set related to slow thinking; a verification module 78 configured to verify the key neuron set by different degrees of unstructured pruning and activation intensity inhibition, and analyze the reasoning performance changes of the large language model; and a fine-tuning module 79 configured to construct a gradient mask matrix based on the position information of the key neurons in the key neuron set, fine-tune the key neurons in a mask manner based on the gradient mask matrix, and obtain an optimized large language model.

[0116] It should be noted that the decoupling device of the slow thinking and fast response areas in the large language model provided in the above embodiment is only exemplified by the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the decoupling device of the slow thinking and fast response areas in the large language model provided in the above embodiment and the decoupling method of the slow thinking and fast response areas in the large language model belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0117] Figure 8 A structural schematic diagram of an electronic device suitable for implementing the embodiments of the present disclosure is shown. It should be noted that Figure 8 The electronic device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0118] As Figure 8 shown, the electronic device includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage portion 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0119] The following components are connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, and the like; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage portion 1008 including a hard disk, and the like; and a communication portion 1009 including a network interface card such as a LAN card, a modem, and the like. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 1010 as needed, so that a computer program read therefrom is installed in the storage portion 1008 as needed.

[0120] The above is only the preferred embodiment of the present application, and it should be noted that those skilled in the art can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for decoupling slow thinking and fast response areas in a large language model, characterized by: include: constructing slow thinking prompts and fast response prompts, inputting the slow thinking prompts and the fast response prompts into a large language model respectively, and recording neuron activation values ​​of each linear layer of the large language model; Calculating an activation value difference vector based on the neuron activation values ​​corresponding to the slow thinking prompt and the fast response prompt, and obtaining a prompt difference vector based on the activation value difference vector; Iteratively processing multiple reasoning problems, obtaining the intersection of the positive value positions of the prompt difference vectors in all reasoning problems, and identifying the key neuron set related to slow thinking; Validating the key neuron set and analyzing the changes in the inference performance of the large language model through different degrees of unstructured pruning and activation intensity suppression; Based on the position information of the key neurons in the key neuron set, a gradient mask matrix is ​​constructed, and the key neurons are fine-tuned in a masking manner based on the gradient mask matrix to obtain an optimized large language model.

2. The method according to claim 1, characterized in that Based on the neuron activation values ​​corresponding to the slow thinking prompt and the fast response prompt respectively, an activation value difference vector is calculated, including: performing a difference operation on each layer of neuron activation values ​​corresponding to the slow thinking prompt and the fast response prompt respectively to obtain the activation value difference vector of each layer.

3. The method according to claim 1, characterized in that Iteratively process multiple reasoning problems, obtain the intersection of the positive value positions of the prompt difference vectors in all reasoning problems, and identify the key neuron sets related to slow thinking, including: For each inference sample, extract all positively activated neuron indices; The intersection of all positively activated neuron indices among multiple inference samples is taken to obtain a set of neuron sets that are always highly activated under the slow thinking prompt, which are used as the key neuron sets related to slow thinking.

4. The method according to claim 1, wherein Based on the position information of the key neurons in the key neuron set, a gradient mask matrix is ​​constructed, including: based on the position information of the key neurons in the key neuron set, the positions corresponding to the position information in the weight matrix of the key neurons in the key neuron set are set to 1, and the remaining positions are set to 0, so as to construct the gradient mask matrix.

5. The method according to claim 4, characterized in that Fine-tuning the key neurons using a masking method based on the gradient mask matrix includes: using a sparse update method, retaining only the gradient updates whose mask positions in the gradient mask matrix are 1, freezing the rest, and writing the updated values ​​into the corresponding weight matrix positions through dynamic mapping to fine-tune the key neurons.

6. The method according to claim 1, characterized in that The key neuron set is verified, and the changes in the reasoning performance of the large language model are analyzed through different degrees of unstructured pruning and activation intensity suppression, including: 50%, 75%, and 100% pruning and activation intensity suppression of the key neuron set, and the reasoning coherence, arithmetic accuracy, and structural integrity of the large language model are evaluated in a multi-step reasoning task.

7. A device for decoupling slow thinking and fast response areas in a large language model, characterized in that: include: a construction module configured to construct slow thinking prompts and fast response prompts, input the slow thinking prompts and the fast response prompts into a large language model, and record neuron activation values ​​of each linear layer of the large language model; a calculation module configured to calculate an activation value difference vector based on the neuron activation values ​​corresponding to the slow thinking prompt and the fast response prompt, and obtain a prompt difference vector based on the activation value difference vector; an identification module configured to iteratively process a plurality of reasoning problems, obtain the intersection of the positive positions of the cue difference vectors in all the reasoning problems, and identify a set of key neurons associated with slow thinking; a verification module configured to verify the key neuron set and analyze the change in reasoning performance of the large language model through different degrees of unstructured pruning and activation intensity suppression; The fine-tuning module is configured to construct a gradient mask matrix based on the position information of the key neurons in the key neuron set, and fine-tune the key neurons in a masking manner based on the gradient mask matrix to obtain an optimized large language model.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. A computer device, characterized in that: include: memory and processor, The memory stores a computer program; The processor is configured to execute a computer program stored in the memory, wherein the computer program enables the processor to execute the method according to any one of claims 1 to 6 when the computer program is executed.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images