Methods, systems, devices and media for enhancing reasoning in large language models

Abstract

This invention provides a method, system, device, and medium for enhancing reasoning in a large language model, relating to the fields of artificial intelligence and natural language processing. The method includes: constructing semantic opposition prompt pairs based on training sample questions and inputting them into a pre-trained large language model to obtain semantic embedding vectors; training a control vector generator based on the sample question control vectors corresponding to the training sample questions, with the goal of minimizing the deviation between the sample question control vectors and the semantic embedding vectors, and obtaining control weight factors for each layer; inputting the question to be processed into the trained control vector generator to obtain a control vector sequence; and performing semantic activation on the Transformer module based on the control vector sequence and control weight factors to obtain the target reasoning result. This invention achieves targeted control of the semantic activation path within the large language model, thereby generating reasoning results with high accuracy.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing technology, and in particular to a method, system, device and medium for enhancing reasoning in large language models. Background Technology

[0002] Large language models have demonstrated powerful capabilities in various natural language tasks, but their reasoning processes are often black boxes, making it difficult to guarantee logical consistency and factual accuracy. Especially in complex reasoning tasks (such as mathematical proofs, multi-hop question answering, and code generation), large language models are prone to producing jumpy, contradictory, or off-topic answers.

[0003] Existing augmentation methods mainly include prompt engineering, chain-of-thought (CoT) fine-tuning, or retrieval augmented generation (RAG). However, these methods either rely on manual design, require a large amount of labeled data, or cannot dynamically control the internal state of the model during the inference stage.

[0004] In recent years, some studies have attempted to guide model behavior by intervening in the output of the Transformer's intermediate layers, but these efforts lack systematic and learnable control mechanisms, making it difficult to adapt to different semantic issues. Therefore, there is an urgent need for a method, system, device, and medium for enhancing reasoning in large language models to address these problems. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method, system, device, and medium for enhancing reasoning in large language models.

[0006] This invention provides a method for enhancing reasoning in large language models, comprising:

[0007] Based on the training sample questions, semantic opposition prompt pairs are constructed, and the semantic opposition prompt pairs are input into a pre-trained large language model to obtain the semantic embedding vectors corresponding to the training sample questions. The semantic embedding vectors include positive semantic embedding vectors and negative semantic embedding vectors.

[0008] Based on the sample problem control vector corresponding to the training sample problem, the control vector generator is trained with the goal of minimizing the deviation between the sample problem control vector and the semantic embedding vector. The trained control vector generator is obtained, and the control weight factor corresponding to each layer in the trained control vector generator is obtained. The semantic embedding vector difference represents the difference between the positive semantic embedding vector and the negative semantic embedding vector.

[0009] The problem to be processed is input into the trained control vector generator to obtain a control vector sequence output by the trained control vector generator, wherein the number of layers of the control vector sequence corresponds to the number of Transformer layers of the pre-trained large language model;

[0010] Based on the control vector sequence and the control weight factor, semantic activation is performed on the Transformer module of the corresponding layer in the pre-trained large language model to obtain the target reasoning result of the problem to be processed output by the pre-trained large language model.

[0011] According to the present invention, a method for enhancing reasoning in a large language model includes constructing semantic opposition prompt pairs based on training sample questions, comprising:

[0012] Based on the training sample questions, construct positive and negative prompt statements;

[0013] Based on the positive and negative prompts, the semantically opposing prompt pair is generated.

[0014] The step of inputting the semantic opposition cue pair into a pre-trained large language model to obtain the semantic embedding vector corresponding to the training sample question includes:

[0015] The positive and negative prompts are input into a pre-trained large language model to obtain the positive semantic embedding vector corresponding to the positive prompt and the negative semantic embedding vector corresponding to the negative prompt.

[0016] According to the present invention, a method for enhancing reasoning in a large language model is provided, wherein the control vector generator is constructed based on a lightweight neural network;

[0017] The process involves training a control vector generator based on the sample problem control vector corresponding to the training sample problem, with the supervision objective of minimizing the deviation between the sample problem control vector and the semantic embedding vector, to obtain a trained control vector generator, including:

[0018] The training sample questions are input into the text encoder to obtain the question embedding vectors corresponding to the training sample questions;

[0019] The problem embedding vector is input into the control vector generator to obtain the sample problem control vector output by the control vector generator;

[0020] Based on the deviation between the sample problem control vector and the semantic embedding vector, a mean squared error loss function is constructed.

[0021] Based on the training sample problem and the mean squared error loss function, the control vector generator is trained to obtain the trained control vector generator.

[0022] According to the large language model inference enhancement method provided by the present invention, the mean squared error loss function is specifically as follows:

[0023] ;

[0024] in, This represents the mean squared error loss value; Indicates the first Layer sample problem control vector; Indicates the first The layer's positive semantic embedding vector; Indicates the first Negative semantic embedding vector of the layer; This represents the total number of Transformer layers in the pre-trained large language model.

[0025] According to the present invention, a method for enhancing inference in a large language model includes obtaining the control weight factors corresponding to each layer in the trained control vector generator, comprising:

[0026] Based on the thought chain data, the trained control vector generator is guided to learn and obtain the control weight factor corresponding to each layer.

[0027] According to the present invention, a method for enhancing reasoning in a large language model is provided, wherein the control vector sequence is , where This represents the total number of Transformer layers in the pre-trained large language model; This represents the dimension of each layer of control vectors.

[0028] According to the present invention, a method for enhancing inference in a large language model includes performing semantic activation on the Transformer module of the corresponding layer in the pre-trained large language model based on the control vector sequence and the control weight factor, comprising:

[0029] Based on a preset control vector injection method, each layer of control vectors and the corresponding control weight factors in the control vector sequence are injected into the Transformer module of the corresponding layer in the pre-trained large language model for semantic activation. The preset control vector injection method includes layer-by-layer addition or multiplication fusion, and the injection position is the self-attention layer of the Transformer module.

[0030] This invention also provides a large language model reasoning enhancement system, comprising:

[0031] A semantic opposition prompt construction module is used to construct semantic opposition prompt pairs based on training sample questions, and input the semantic opposition prompt pairs into a pre-trained large language model to obtain the semantic embedding vectors corresponding to the training sample questions, wherein the semantic embedding vectors include positive semantic embedding vectors and negative semantic embedding vectors;

[0032] The control vector generator training module is used to train the control vector generator based on the sample problem control vector corresponding to the training sample problem, with the goal of minimizing the deviation between the sample problem control vector and the semantic embedding vector. The module obtains the trained control vector generator and acquires the control weight factor corresponding to each layer in the trained control vector generator. The semantic embedding vector difference represents the difference between the positive semantic embedding vector and the negative semantic embedding vector.

[0033] The control vector sequence generation module is used to input the problem to be processed into the trained control vector generator to obtain the control vector sequence output by the trained control vector generator, wherein the number of layers of the control vector sequence corresponds to the number of Transformer layers of the pre-trained large language model;

[0034] The reasoning enhancement module is used to perform semantic activation on the Transformer module of the corresponding layer in the pre-trained large language model based on the control vector sequence and the control weight factor, so as to obtain the target reasoning result of the problem to be processed output by the pre-trained large language model.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the large language model reasoning enhancement method as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large language model reasoning enhancement method as described above.

[0037] The large language model reasoning enhancement method, system, device, and medium provided by this invention constructs semantic opposition prompts to learn control vectors and injects hierarchical control vectors into the Transformer module during the reasoning stage, thereby achieving targeted control of the semantic activation path within the large language model and generating reasoning results with high accuracy. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 A flowchart illustrating the large language model reasoning enhancement method provided by this invention;

[0040] Figure 2 One of the schematic diagrams of the training process provided by the present invention;

[0041] Figure 3 The second schematic diagram of the training process provided by the present invention;

[0042] Figure 4 A schematic diagram of the reasoning process provided by the present invention;

[0043] Figure 5 This is a schematic diagram of the structure of the large language model reasoning enhancement system provided by the present invention;

[0044] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0046] Figure 1 This is a flowchart illustrating the large language model reasoning enhancement method provided by the present invention, as shown below. Figure 1 As shown, this invention provides a method for enhancing reasoning in large language models, including:

[0047] Step 101: Based on the training sample question, construct a semantic opposition prompt pair and input the semantic opposition prompt pair into the pre-trained large language model to obtain the semantic embedding vector corresponding to the training sample question, wherein the semantic embedding vector includes a positive semantic embedding vector and a negative semantic embedding vector.

[0048] In this invention, for each training sample problem (which can be set as a problem) q We need to construct semantically contrasting cue pairs, denoted as follows: and .For example, It could be "Step-by-step correct solution:..." (correct and reasonable solution direction). This could be "Common Incorrect Solutions:..." (incorrect, unreasonable solutions). This approach provides hints about the problem from both positive and negative perspectives, facilitating the subsequent acquisition of embedding vectors with different semantic orientations.

[0049] Then, the constructed semantic opposition prompts are... and The inputs are fed into the pre-trained large language model. M In China, for pre-trained large language models... M Each layer (let the first layer) layer, =1, 2, 3, ... N Extract its hidden state. Among them, by The obtained hidden state is denoted as the positive embedding, and is derived from... The obtained hidden state is denoted as the negative embedding, thus yielding the training sample problem. q The corresponding semantic embedding vectors, including positive semantic embedding vectors and negative semantic embedding vector .

[0050] Step 102: Based on the sample problem control vector corresponding to the training sample problem, with the goal of minimizing the deviation between the sample problem control vector and the semantic embedding vector, train the control vector generator to obtain the trained control vector generator, and obtain the control weight factor corresponding to each layer in the trained control vector generator, wherein the semantic embedding vector difference represents the difference between the positive semantic embedding vector and the negative semantic embedding vector.

[0051] In this invention, the training sample problem is addressed. q The question embedding vector is obtained by processing the text using a text encoder. z q Then, the question is embedded into the vector. z q Input to control vector generator G In the process of generating sample problem control vectors ,in, This indicates the number of layers in the large language model, meaning that a corresponding sample question control vector is generated for each layer.

[0052] Furthermore, using positive semantic embedding vectors and negative semantic embedding vector The difference between them (i.e., the semantic embedding vector difference) As the monitoring target, the loss function (mean squared error loss function) is defined as follows:

[0053]

[0054] The loss function described above is used to measure the sample problem control vector. The deviation between the semantic embedding vector and the difference. This represents the mean squared error loss value; Indicates the first Layer sample problem control vector; Indicates the first The layer's positive semantic embedding vector; Indicates the first Negative semantic embedding vector of the layer; This represents the total number of Transformer layers in the pre-trained large language model. This invention optimizes the parameters of the control vector generator using the aforementioned mean squared error loss function.

[0055] Then, with minimizing this bias as the supervision objective, the control vector generator is... G The parameters are optimized during training. The pre-training of the control vector generator is completed by repeating the above process of constructing semantic contrastive prompt pairs and training the control vector generator on a large-scale question-answering dataset (i.e., repeating steps 101 and 102).

[0056] Furthermore, on the data labeled with thought chains, for each layer... Learn a control weight factor Control weighting factor The values ​​of these weights range from [0, 1]. The purpose of learning these weight factors is to enable key inference layers to obtain higher control weights, so that different layers of the model can be more effectively controlled in subsequent inference processes, highlighting the role of key inference steps.

[0057] Step 103: Input the problem to be processed into the trained control vector generator to obtain the control vector sequence output by the trained control vector generator, wherein the number of layers of the control vector sequence corresponds to the number of Transformer layers of the pre-trained large language model.

[0058] In this invention, when there is a new problem to be solved (let's say...) q test When doing so, first address the issues to be processed. q test Encode to obtain the problem embedding z test Then, embed the question. z testInput to the pre-trained control vector generator G In the middle, control vector generator G It will output a sequence of control vectors { c 1 , c 2 , c 3 ... c N Furthermore, the number of layers in this control vector sequence corresponds to the number of Transformer layers in the pre-trained large language model, meaning that each layer has a corresponding control vector for subsequent adjustment of that layer.

[0059] Step 104: Based on the control vector sequence and the control weight factor, perform semantic activation on the Transformer module of the corresponding layer in the pre-trained large language model to obtain the target reasoning result of the problem to be processed output by the pre-trained large language model.

[0060] In this invention, based on the control vector sequence and control weight factors obtained from the above steps, semantic activation is performed on the Transformer module of the corresponding layer in the pre-trained large language model. Specifically, the weighted control vectors are... Injected into the large language model In the Transformer module of the layer. The injection point is after the self-attention output and before the residual connection, and its calculation formula is:

[0061]

[0062] in, It is the first The output of each layer. In this way, information from the control vector is incorporated into the computation of each layer of the model, thereby dynamically controlling the direction of the inference process.

[0063] In this invention, the large language model, under the control of the aforementioned control vectors, completes the forward reasoning process. During this process, the large language model reasons in a structured manner, forming a structured thought process, and ultimately outputs a high-quality, structured, and highly consistent target reasoning result—the answer to the problem to be solved.

[0064] The large language model reasoning enhancement method provided by this invention constructs semantic opposition cue pairs to learn control vectors and injects hierarchical control vectors into the Transformer module during the reasoning stage, thereby achieving targeted control of the semantic activation path within the large language model and generating reasoning results with high accuracy.

[0065] Based on the above embodiments, the step of constructing semantic contrastive cue pairs based on training sample questions includes:

[0066] Based on the training sample questions, construct positive and negative prompt statements;

[0067] Based on the positive and negative prompts, the semantically opposing prompt pair is generated.

[0068] The step of inputting the semantic opposition cue pair into a pre-trained large language model to obtain the semantic embedding vector corresponding to the training sample question includes:

[0069] The positive and negative prompts are input into a pre-trained large language model to obtain the positive semantic embedding vector corresponding to the positive prompt and the negative semantic embedding vector corresponding to the negative prompt.

[0070] In this invention, it is first necessary to clarify the specific content and core points of the training sample problem. For example, if the training sample problem is a question about mathematical operation rules: how to calculate the multiplication of two-digit numbers, then the prompts should be constructed around the core knowledge of this problem.

[0071] Positive prompts aim to guide the model in thinking and answering in a correct, reasonable, and expected direction. They typically include information such as correct methods, perspectives, and steps. For example, regarding "how to calculate two-digit multiplication," a positive prompt might be "calculate two-digit multiplication by following the correct reasoning steps: multiply the units digits first, then the tens digits, and finally add the two results." This clarifies the correct calculation process, providing a correct semantic direction for the large language model.

[0072] Negative hints, in contrast to positive hints, focus on incorrect, unreasonable, or unexpected situations. They help the model learn a semantic difference space, that is, the distinction between correct and incorrect. For the question "How to calculate the multiplication of two-digit numbers," a negative hint could be "Common incorrect reasoning: directly adding two two-digit numbers as one-digit numbers." This statement presents a common incorrect calculation method, contrasting sharply with the positive hint, allowing the large language model to clearly distinguish between correct and incorrect semantics.

[0073] In this invention, semantic opposition refers to two statements that are semantically opposite, relative, or contradictory. Positive and negative prompts are two semantically opposing statements addressing the same training sample problem. Furthermore, positive and negative prompts are combined to form semantically opposing prompt pairs. For example, for the problem of calculating two-digit multiplication, the generated semantically opposing prompt pair would be "(Calculate two-digit multiplication according to the correct reasoning steps: multiply the units digit first, then the tens digit, and finally add the two results)" and "(Common incorrect reasoning: directly adding two two-digit numbers as one-digit numbers)". This prompt pair clearly demonstrates the two different semantic directions of correct and incorrect approaches to the same problem, providing a foundation for subsequent model learning of semantic differences.

[0074] In this invention, the pre-trained large language model possesses powerful language understanding and feature extraction capabilities, enabling deep processing of input text. When positive prompts are input into the pre-trained large language model, the model encodes and processes them, extracting their semantic features at various levels. Ultimately, the model outputs a vector representing the semantic information of the positive prompt; this vector is the positive semantic embedding vector. For example, for the positive prompt "Calculate two-digit multiplication by following the correct reasoning steps of multiplying the units digit first, then the tens digit, and finally adding the two results," the model transforms it into a vector of a specific dimension. This vector contains the semantic features of the correct calculation method implied by the positive prompt.

[0075] Correspondingly, when the negative prompt is input into the pre-trained large language model, the model will process it accordingly, extract its semantic features, and output a vector, namely the negative semantic embedding vector. Taking the negative prompt "Common incorrect reasoning: directly adding two two-digit numbers as one-digit numbers to calculate multiplication" as an example, the model will generate a vector reflecting the semantic features of this incorrect calculation method.

[0076] Through the above steps, semantic opposition prompt pairs can be constructed based on the training sample questions, and corresponding positive and negative semantic embedding vectors can be obtained using a pre-trained large language model. This provides an important data foundation for the subsequent model to learn the semantic difference space and perform more accurate reasoning.

[0077] Based on the above embodiments, the control vector generator is constructed based on a lightweight neural network;

[0078] The process involves training a control vector generator based on the sample problem control vector corresponding to the training sample problem, with the supervision objective of minimizing the deviation between the sample problem control vector and the semantic embedding vector, to obtain a trained control vector generator, including:

[0079] The training sample questions are input into the text encoder to obtain the question embedding vectors corresponding to the training sample questions;

[0080] The problem embedding vector is input into the control vector generator to obtain the sample problem control vector output by the control vector generator;

[0081] Based on the deviation between the sample problem control vector and the semantic embedding vector, a mean squared error loss function is constructed.

[0082] Based on the training sample problem and the mean squared error loss function, the control vector generator is trained to obtain the trained control vector generator.

[0083] In this invention, the control vector generator is constructed based on a lightweight neural network. Lightweight neural networks have the characteristics of relatively few parameters and low computational complexity, which enables more efficient operation while ensuring certain performance, reducing the consumption of computing resources. They are suitable for scenarios that require rapid generation of control vectors and can be easily integrated into existing model architectures.

[0084] Specifically, this invention first inputs the training sample questions into a text encoder. The text encoder's role is to encode the input text, converting it into a numerical vector form that a computer can understand and process. For example, a common text encoder could be a pre-trained word embedding model or a more complex Transformer-based encoder.

[0085] The text encoder encodes each word or subword in the training sample question, and then combines or aggregates these encodings to obtain a fixed-dimensional vector. This vector is the question embedding vector corresponding to the training sample question, which contains the semantic information of the training sample question and is the basis for the subsequent generation of control vectors.

[0086] Next, the question embedding vector obtained in the previous step is input into the control vector generator. Since the control vector generator is a lightweight neural network structure, it can accept the question embedding vector as input and perform a series of non-linear transformations on the input vector through its internal neural network layers. After processing by the control vector generator, it outputs a vector that matches the dimension of the Transformer layer; this vector is the sample question control vector. The dimension of the sample question control vector corresponds to the dimension of the Transformer layer in the large language model, the purpose of which is to enable interaction with the Transformer layer and control the model's inference process.

[0087] In this invention, positive semantic embedding vectors are obtained by constructing training sample questions into semantically contrasting cue pairs (such as positive and negative cue pairs) and inputting them into a pre-trained large language model. and negative semantic embedding vector The semantic embedding vector difference is... , represents the difference space between positive and negative semantics, reflecting the distance between the correct and incorrect semantic directions.

[0088] Furthermore, based on the difference between the sample problem control vector C and the semantic embedding vector ( The mean squared error loss function is constructed based on the deviation between the two values. By minimizing this loss function, the generated control vector C can be made as close as possible to the semantic embedding vector difference, thus enabling the control vector to better capture information from the semantic difference space.

[0089] In this invention, based on the training sample problem and the constructed mean squared error loss function, optimization algorithms (such as stochastic gradient descent, Adam, etc.) are used to update and optimize the parameters of the control vector generator. During the training process, the neural network weights inside the control vector generator are continuously adjusted so that the value of the loss function gradually decreases, that is, the generated control vector becomes closer and closer to the semantic embedding vector difference.

[0090] The training process described above (i.e., continuously inputting different training sample questions to train the control vector generator) is repeated on a large-scale question-answering dataset until the performance of the control vector generator reaches a stable level and meets the requirements. Training is then stopped, resulting in a well-trained control vector generator. This trained generator can quickly generate appropriate control vectors based on the input question embedding vector, which are then used to regulate the inference process of the pre-trained large language model.

[0091] In this invention, after obtaining the trained control vector generator, to further optimize the control effect on the pre-trained large language model, it can be trained using Chain of Thought (CoT) data to learn the control weight factors for each layer. CoT data contains detailed steps and logic of question reasoning. By analyzing this data, it can be understood which layers are more critical for generating the correct answer in different question reasoning processes. Therefore, this invention can learn a weight factor for each layer, enabling key reasoning layers to obtain higher regulatory weights, thereby more effectively dynamically regulating the reasoning process of the large language model and improving the quality and consistency of the model's output answers.

[0092] Based on the above embodiments, obtaining the control weight factor corresponding to each layer in the trained control vector generator includes:

[0093] Based on the thought chain data, the trained control vector generator is guided to learn and obtain the control weight factor corresponding to each layer.

[0094] In this invention, based on the already trained control vector generator, the control vector generator's effect on the regulation of each layer of the pre-trained large language model is further optimized.

[0095] The thought chain data records the detailed process of problem reasoning, including a series of intermediate reasoning steps from the initial question to the conclusion, clearly presenting the logical chain of reasoning. This invention utilizes thought chain data to guide the learning of the control vector generator, enabling the control vector to have higher regulatory weights at key nodes in the reasoning chain, such as intermediate reasoning steps and the conclusion generation layer. This strengthens the logical coherence of the entire reasoning process and improves the quality of the model's output answer.

[0096] Specifically, this invention collects a large number of question-and-answer datasets with thought chain annotations. Each question in these datasets is accompanied by detailed annotations of the reasoning process, covering the complete logical chain from question understanding, information extraction, intermediate reasoning steps to the generation of the final conclusion. For example, for a mathematical problem, the annotations may include intermediate reasoning steps such as listing known conditions, performing step-by-step calculations using relevant formulas, and checking the calculation results, as well as the final conclusion generation steps.

[0097] Then, the collected thought chain labeled data is preprocessed to ensure that the data format is uniform and accurate, including cleaning the text to remove irrelevant characters and noise; numbering or segmenting the reasoning steps for subsequent analysis and processing; and converting the data into a format suitable for model input, such as converting text into vector representation.

[0098] Furthermore, based on the characteristics of the thought chain data and the general rules of the reasoning process, the types of key nodes in the reasoning chain are determined. Typically, intermediate reasoning steps and the conclusion generation layer are considered key nodes. Intermediate reasoning steps serve as a bridge connecting the initial state of the problem and the final conclusion, containing the core logic and key information for solving the problem; the conclusion generation layer is the final output of the entire reasoning process, and its accuracy directly affects the quality of the answer. In the labeled thought chain data, the identified key nodes are marked. For example, special labels or identifiers can be added to each intermediate reasoning step and conclusion generation layer to accurately identify and locate these key nodes in subsequent learning processes.

[0099] In this invention, a model for learning weight factors is constructed based on a pre-trained control vector generator. This model embeds the input question into a vector and a control vector, while also considering key node information from the thought chain annotation data. A specific algorithm and loss function are used to adjust the weight factors of each layer. During the learning process, the model analyzes the importance of each layer in the reasoning process based on the key node information in the thought chain annotation data. For key reasoning layers (such as those corresponding to intermediate reasoning steps and conclusion generation layers), higher weight factors are tended to be assigned, resulting in stronger control vector regulation in these layers; while for non-key layers, the weight factors are relatively lower. Through multiple iterations of training, the weight factors are continuously updated until the model's performance stabilizes and meets the requirements. In each iteration, a control vector is generated based on the current weight factors, and the weight factors are adjusted according to the thought chain annotation data and the loss function, gradually optimizing the model's regulation effect on each layer.

[0100] After the above supervised learning process, when the model's performance stabilizes, the results for each layer can be obtained. Corresponding control weight factor Control weighting factor This reflects the importance of each layer in the reasoning process. Key reasoning layers will receive higher control weights, enabling the control vector generator to more effectively regulate the pre-trained large language model and strengthen the logical coherence of the reasoning process.

[0101] Based on the above embodiments, the control vector sequence is as follows: , where This represents the total number of Transformer layers in the pre-trained large language model; This represents the dimension of each layer of control vectors.

[0102] In this invention, the control vector sequence is presented as a The vector matrix can systematically organize and store information, facilitating precise control of each layer of the pre-trained large language model. Each row and column in the vector matrix carries specific semantics and functions, collectively forming a dynamic control vector for the model's inference process.

[0103] In large language models, the Transformer architecture is a core component, consisting of multiple stacked layers. In this invention, N This represents the total number of Transformer layers in the pre-trained large language model. Each layer contains sub-modules such as self-attention mechanisms and feedforward neural networks, which are responsible for different levels of feature extraction and transformation of the input data.

[0104] In this invention, each row of the control vector sequence corresponds to a specific layer in the large language model. This one-to-one correspondence ensures that each control vector precisely targets the corresponding layer of the model, enabling layered control of the model's inference process. For example, if the pre-trained large language model has 12 Transformer layers, then the control vector sequence is a 12×... d The matrix contains 12 rows of vectors that correspond to the 12 layers of the model.

[0105] In this invention, the control vector is a signal carrier used to regulate the various layers of the pre-trained large language model, and its dimension is... d The dimensionality determines the amount of information a vector can carry. Higher dimensionality allows for a richer representation of features and more precise control over model layers. The specific value needs to be determined by considering factors such as model complexity, task requirements, and computational resources. The dimensionality of the vector is controlled by each layer. d It needs to match or be compatible with the feature dimensions of the corresponding layer of the pre-trained large language model to ensure that the control vector can effectively interact with the model layer.

[0106] The control vector sequence is obtained by inputting the problem embedding into a lightweight neural network generator. Lightweight neural network generators are characterized by high computational efficiency and few parameters, enabling them to quickly convert problem embeddings into control vector sequences. Specifically, the problem embedding is first input into the control vector generator, which transforms the problem embedding into a sequence through a series of nonlinear transformations and calculations. The vector matrix, i.e., the control vector sequence, is generated dynamically, capable of generating corresponding control vector sequences based on different problem embeddings, thereby achieving adaptive control of the pre-trained large language model.

[0107] In practical applications, The control vector sequence provides a more flexible and precise control method for pre-trained large language models. By setting different control vectors for different layers, the model can be guided to focus on different features and information during inference, thereby improving the model's performance on specific tasks. For example, in question-answering tasks, by adjusting the control vector sequence, the model can focus more on logical reasoning in the intermediate inference layer and more on the accuracy and completeness of the answer in the conclusion generation layer, thus generating more reasonable and accurate answers.

[0108] Based on the above embodiments, the step of performing semantic activation on the Transformer module of the corresponding layer in the pre-trained large language model based on the control vector sequence and the control weight factor includes:

[0109] Based on a preset control vector injection method, each layer of control vectors and the corresponding control weight factors in the control vector sequence are injected into the Transformer module of the corresponding layer in the pre-trained large language model for semantic activation. The preset control vector injection method includes layer-by-layer addition or multiplication fusion, and the injection position is the self-attention layer of the Transformer module.

[0110] In this invention, control vectors and their control weight factors are injected into specific positions of a pre-trained large language model to guide the large language model in the correct direction during the reasoning process, activate the task-related semantic information in the model, thereby improving the quality of the model's output results, making the reasoning process more logical and interpretable, and the results more accurate and reliable.

[0111] Each control vector in the control vector sequence carries specific regulatory information, used to adjust the semantic representation of the corresponding layer of the model. The control weight factor, on the other hand, measures the importance of each control vector in the regulatory process and corresponds one-to-one with each control vector. By combining control vectors and control weight factors, dynamic and precise regulation of the semantic representation of each layer of the model can be achieved.

[0112] In this invention, the preset control vector injection method can be a layer-by-layer additive fusion method, which involves adding the weighted control vector to a specific output of the Transformer module element-by-element. Specifically, for each layer of the pre-trained large language model, the control vector is first weighted according to the control weight factor corresponding to that layer, and then the weighted control vector is added element-by-element to the self-attention output or feedforward network input of the Transformer module of that layer. This fusion method achieves dynamic shifting of semantic representation. By adding the control vector to the output of the model layer, additional semantic information can be introduced or the strength and direction of the original semantics can be adjusted without changing the original computational flow of the model. For example, in question-answering tasks, if it is desired that the model pay more attention to the key information in the question, the control vector emphasizing the key information can be injected into the model layer through additive fusion, so that the model focuses more on the semantic representation related to the key information in subsequent calculations, thereby guiding the reasoning process in the correct direction.

[0113] Another preset control vector injection method is multiplicative fusion, which multiplies the weighted control vector element-wise with a specific output of the Transformer module. Similarly, the control vector is first weighted according to the control weight factor, and then multiplied with the model layer's output. Multiplicative fusion can scale and filter semantic representations. By multiplying with the control vector, it can amplify or reduce semantic components related to the control vector in the model layer's output, while suppressing irrelevant semantic components. For example, in some tasks, the model may need to ignore some interfering information and only focus on semantic features highly relevant to the task. In this case, multiplicative fusion can inject control vectors that suppress interfering information into the model layer, allowing the model to focus more on key semantics during inference, improving the accuracy and efficiency of inference.

[0114] Self-attention is a core component of the Transformer module, capable of capturing dependencies between different positions in the input sequence, enabling large language models to consider global information when processing sequence data. In the self-attention layer, the large language model calculates attention weights to determine the degree of attention each position gives to other positions, thereby generating more accurate and richer semantic representations. This invention injects control vectors and control weight factors into the self-attention layer because this position allows direct intervention in how the model focuses on the input sequence. By adjusting the output of the self-attention layer, the large language model can be guided to focus more on task-related information during inference, avoiding interference from irrelevant information and thus constructing a more reasonable and accurate inference chain. For example, in text generation tasks, injecting control vectors into the self-attention layer allows the large language model to pay more attention to contextual consistency and logic when generating text, avoiding jumpy or divergent outputs and ensuring that the inference process is interpretable and the results are verifiable.

[0115] This invention, through the aforementioned injection method and location, enables control vectors and control weight factors to effectively activate the semantic information of corresponding layers in the pre-trained large language model. Guided by the control vectors, the model gradually constructs a structured reasoning flow and a structured thought flow. The structured reasoning flow ensures that the model gradually builds a reasoning chain during the reasoning process, avoiding jumps or divergent outputs, making the reasoning process interpretable and the results verifiable. The structured thought flow, guided by the control vectors, enables the model to gradually construct intermediate states that conform to logical rules, avoiding divergent or contradictory outputs, and ultimately generating interpretable and highly accurate answers.

[0116] Figure 2 One of the schematic diagrams of the training process provided by the present invention can be referred to. Figure 2The diagram illustrates a training process for a Large Language Model (LLM). The primary goal is to generate control vectors to guide the model's inference direction, thereby improving the quality and logical consistency of the model's output. The following is a detailed explanation of the training steps:

[0117] Input prompt:

[0118] Positive Prompt: Enter a question in the form of {Question} and it will serve as a positive prompt.

[0119] Negative Prompt: The same question is input in the form of {Question}, but as a negative sample prompt. Positive and negative samples will be given different semantic guidance in subsequent processing.

[0120] The Large Language Model (LLM) consists of multiple (N) Transformer layers. Positive and negative prompts are input into the LLM, and after processing by these Transformer layers, corresponding positive and negative embeddings are generated. These embedding vectors capture the feature representations of the question in different semantic directions.

[0121] In this invention, the question embedding is input into a control vector generator. The control vector generator is a lightweight neural network that generates a control vector based on the input question embedding. The dimension of this control vector is related to the Transformer layer of the large language model.

[0122] Furthermore, the difference between positive and negative sample embeddings (Positive Embeddings - Negative Embeddings) is calculated. The mean squared error (MSE) loss function is then used to calculate the error between the control vector and this difference. The goal of the loss function is to minimize the deviation between the control vector and the difference between the positive and negative sample embeddings, i.e., MSE Loss: Min(MSE(Control Vector, Positive Embeddings - Negative Embeddings)). By minimizing this loss function, the control vector generator is trained so that the generated control vectors can better capture the semantic differences between positive and negative samples.

[0123] After training, in practical applications, control vectors and control weights are injected into the corresponding Transformer layers of the large language model. The injection points are typically after the self-attention layer and before the residual connections. The control vectors are combined with the model's output through layer-by-layer addition or multiplication, enabling dynamic adjustment of the semantic representation. This guides the model in the correct direction during reasoning, constructing a structured reasoning and thought process, ultimately generating high-quality, interpretable answers.

[0124] Figure 3 This is the second schematic diagram of the training process provided by the present invention, which can be referred to. Figure 3 As shown, the weight factors of each layer are learned to optimize the control vector's effect on model inference. The following is a detailed explanation of the training steps:

[0125] Input Question: The question is fed into the training system of the Large Language Model (LLM) as the initial input information.

[0126] Control Vector Generation: Simultaneously, the question is passed to the control vector generator. The control vector generator is a specific component that generates a corresponding control vector based on the input question. This control vector is used to guide the inference direction of the large language model.

[0127] Weight Factor Learning Process: In the weight bias layer, weight factors are learned for each layer. For each layer of the large language model (from Transformer Layer 1 to Transformer Layer N), a corresponding weight factor is learned, labeled α_1, α_2, ..., α_N. These weight factors typically range from [0, 1] and are used to measure the importance or regulatory strength of the control vector in each layer. The control vector generated by the control vector generator interacts with these weight factors. Each weight factor adjusts the corresponding control vector, allowing the strength of the control vector's effect in different layers to be dynamically adjusted according to actual needs.

[0128] The weighted control vector information is injected into the corresponding layer of the large language model. The large language model consists of multiple Transformer layers, which perform complex calculations and processing on the input question and the injected control vector information. After receiving the weighted control vector, each Transformer layer combines it with its own calculation process, such as by element-wise addition or multiplication, to dynamically adjust the semantic representation, thereby affecting the model's understanding and reasoning process.

[0129] The final layer of the large language model outputs the final semantic feature representation, completing Supervised Fine-Tuning (SFT). Throughout the training process, by continuously adjusting the parameters of the control vector generator and the weight factors of each layer, the large language model can better reason based on the guidance of the control vectors, ultimately generating high-quality results that meet expectations. The goal of training is to enable the large language model, under the influence of control vectors, to more accurately understand the question and generate reasonable answers.

[0130] Figure 4 A schematic diagram of the reasoning process provided by the present invention can be referred to. Figure 4 As shown, the core of the large language model inference process guided by control vectors lies in using a pre-trained control vector generator and weight factors to dynamically adjust the computation of the large language model at different layers, in order to generate inference results that are more logical and expected. The following is a detailed explanation of the inference steps:

[0131] Input question: The question to be reasoned is input into the entire reasoning system as the initial input information.

[0132] Control Vector Generation: The question is passed to the control vector generator. The control vector generator generates a corresponding control vector based on the input question. This control vector generator is based on patterns and knowledge learned during previous training and is used to guide the large language model to focus on the correct direction and key information during inference.

[0133] Weighted Control Vectors: The generated control vectors are passed to the corresponding control weight factors. For each layer of the large language model (from Transformer Layer 1 to Transformer Layer N), there is a pre-learned control weight factor. These control weight factors adjust the control vectors by weighting them, so that the strength of the control vectors in different layers can be dynamically allocated according to the actual inference needs. For example, for key inference layers, higher weight factors may be assigned, making the control vectors more effective in that layer.

[0134] Injecting Control Vectors and Inference: The weighted control vectors are injected into the corresponding layers of the large language model. The large language model consists of multiple Transformer layers, and each layer, upon receiving the weighted control vectors, combines them with its own computation process. In this invention, the control vectors are fused with the model's output through element-wise addition or multiplication, thereby dynamically adjusting the semantic representation. This adjustment affects the model's understanding and reasoning process, guiding the model to gradually construct intermediate states that conform to logical rules.

[0135] Output reasoning result: The final layer of the large language model integrates the computational results of all layers and outputs the final reasoning result. This reasoning result is generated under the guidance of control vectors, has better logical coherence and accuracy, and can answer the input question more accurately.

[0136] This invention achieves explicit intervention in the internal reasoning flow of a large language model through control vectors, avoiding the problems of decreased generalization ability and increased computational overhead caused by traditional fine-tuning or hint engineering. Simultaneously, the generation of control vectors depends on knowledge content, enabling the large language model to dynamically adjust its reasoning strategy according to different contexts, enhancing its ability to handle complex problems across multiple domains and types. Furthermore, by injecting control vectors in a layered manner, this invention can precisely control the level of semantic abstraction at different levels while maintaining the original architecture of the large language model, thereby achieving superior performance compared to baseline models in tasks such as logical reasoning, fact verification, and causal inference.

[0137] The large language model reasoning enhancement system provided by this invention is described below. The large language model reasoning enhancement system described below can be referred to in correspondence with the large language model reasoning enhancement method described above.

[0138] Figure 5 This is a schematic diagram of the structure of the large language model reasoning enhancement system provided by the present invention, as shown below. Figure 5As shown, this invention provides a large language model reasoning enhancement system, including a semantic opposition prompt construction module 501, a control vector generator training module 502, a control vector sequence generation module 503, and a reasoning enhancement module 504. The semantic opposition prompt construction module 501 constructs semantic opposition prompt pairs based on training sample questions and inputs these pairs into a pre-trained large language model to obtain semantic embedding vectors corresponding to the training sample questions. These semantic embedding vectors include positive and negative semantic embedding vectors. The control vector generator training module 502 trains a control vector generator based on the sample question control vectors corresponding to the training sample questions, with the goal of minimizing the deviation between the sample question control vector and the semantic embedding vector, to obtain a trained control vector generator. A vector generator is used to obtain the control weight factor corresponding to each layer in the trained control vector generator, wherein the semantic embedding vector difference represents the difference between the positive semantic embedding vector and the negative semantic embedding vector; a control vector sequence generation module 503 is used to input the problem to be processed into the trained control vector generator to obtain a control vector sequence output by the trained control vector generator, wherein the number of layers of the control vector sequence corresponds to the number of Transformer layers of the pre-trained large language model; a reasoning enhancement module 504 is used to perform semantic activation on the Transformer module of the corresponding layer in the pre-trained large language model based on the control vector sequence and the control weight factor to obtain the target reasoning result of the problem to be processed output by the pre-trained large language model.

[0139] The large language model reasoning enhancement system provided by this invention constructs semantic opposition cue pairs to learn control vectors and injects hierarchical control vectors into the Transformer module during the reasoning stage, thereby achieving targeted control of the semantic activation path within the large language model and generating reasoning results with high accuracy.

[0140] The system provided in this embodiment of the invention is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0141] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6As shown, the electronic device may include: a processor 601, a communications interface 602, a memory 603, and a communication bus 604. The processor 601, communications interface 602, and memory 603 communicate with each other via the communication bus 604. The processor 601 can call logical instructions in the memory 603 to execute a large language model inference enhancement method. This method includes: constructing semantic opposition prompt pairs based on training sample questions, and inputting the semantic opposition prompt pairs into a pre-trained large language model to obtain semantic embedding vectors corresponding to the training sample questions. The semantic embedding vectors include positive semantic embedding vectors and negative semantic embedding vectors. Based on the sample question control vectors corresponding to the training sample questions, a control vector generator is trained with the goal of minimizing the deviation between the sample question control vector and the semantic embedding vector, resulting in a trained control vector generator. The trained control vector generator is then obtained. The control weight factor corresponds to each layer in the generator, where the semantic embedding vector difference represents the difference between the positive semantic embedding vector and the negative semantic embedding vector; the problem to be processed is input into the trained control vector generator to obtain a control vector sequence output by the trained control vector generator, wherein the number of layers in the control vector sequence corresponds to the number of Transformer layers in the pre-trained large language model; based on the control vector sequence and the control weight factor, semantic activation is performed on the Transformer module of the corresponding layer in the pre-trained large language model to obtain the target reasoning result of the problem to be processed output by the pre-trained large language model.

[0142] Furthermore, the logical instructions in the aforementioned memory 603 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the large language model inference enhancement method provided by the above methods, the method comprising: constructing semantic opposition cue pairs based on training sample questions, and inputting the semantic opposition cue pairs into a pre-trained large language model to obtain semantic embedding vectors corresponding to the training sample questions, wherein the semantic embedding vectors include positive semantic embedding vectors and negative semantic embedding vectors; based on the sample question control vectors corresponding to the training sample questions, with the goal of minimizing the deviation between the sample question control vectors and the semantic embedding vectors, adjusting the control vectors... The generator is trained to obtain a trained control vector generator, and the control weight factor corresponding to each layer in the trained control vector generator is obtained. The semantic embedding vector difference represents the difference between the positive semantic embedding vector and the negative semantic embedding vector. The problem to be processed is input into the trained control vector generator to obtain a control vector sequence output by the trained control vector generator. The number of layers in the control vector sequence corresponds to the number of Transformer layers in the pre-trained large language model. Based on the control vector sequence and the control weight factor, semantic activation is performed on the Transformer module of the corresponding layer in the pre-trained large language model to obtain the target inference result of the problem to be processed output by the pre-trained large language model.

[0144] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the large language model inference enhancement method provided in the above embodiments. The method includes: constructing semantic opposition cue pairs based on training sample questions, and inputting the semantic opposition cue pairs into a pre-trained large language model to obtain semantic embedding vectors corresponding to the training sample questions, wherein the semantic embedding vectors include positive semantic embedding vectors and negative semantic embedding vectors; and training a control vector generator based on the sample question control vectors corresponding to the training sample questions, with the goal of minimizing the deviation between the sample question control vectors and the semantic embedding vectors, to obtain trained control vectors. A generator is used to obtain the control weight factor corresponding to each layer in the trained control vector generator, wherein the semantic embedding vector difference represents the difference between the positive semantic embedding vector and the negative semantic embedding vector; the problem to be processed is input into the trained control vector generator to obtain the control vector sequence output by the trained control vector generator, wherein the number of layers of the control vector sequence corresponds to the number of Transformer layers of the pre-trained large language model; based on the control vector sequence and the control weight factor, semantic activation is performed on the Transformer module of the corresponding layer in the pre-trained large language model to obtain the target reasoning result of the problem to be processed output by the pre-trained large language model.

[0145] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for enhancing reasoning in a large language model, characterized in that, include: Based on the training sample questions, semantic opposition prompt pairs are constructed, and the semantic opposition prompt pairs are input into a pre-trained large language model to obtain the semantic embedding vectors corresponding to the training sample questions. The semantic embedding vectors include positive semantic embedding vectors and negative semantic embedding vectors. Based on the sample problem control vector corresponding to the training sample problem, the control vector generator is trained with the goal of minimizing the deviation between the sample problem control vector and the semantic embedding vector. The trained control vector generator is obtained, and the control weight factor corresponding to each layer in the trained control vector generator is obtained. The semantic embedding vector difference represents the difference between the positive semantic embedding vector and the negative semantic embedding vector. The problem to be processed is input into the trained control vector generator to obtain a control vector sequence output by the trained control vector generator, wherein the number of layers of the control vector sequence corresponds to the number of Transformer layers of the pre-trained large language model; Based on the control vector sequence and the control weight factor, semantic activation is performed on the Transformer module of the corresponding layer in the pre-trained large language model to obtain the target reasoning result of the problem to be processed output by the pre-trained large language model. The step of semantic activation of the Transformer module of the corresponding layer in the pre-trained large language model based on the control vector sequence and the control weight factor includes: Based on a preset control vector injection method, each layer of control vectors and the corresponding control weight factors in the control vector sequence are injected into the Transformer module of the corresponding layer in the pre-trained large language model for semantic activation. The preset control vector injection method includes layer-by-layer addition or multiplication fusion, and the injection position is the self-attention layer of the Transformer module.

2. The large language model reasoning enhancement method according to claim 1, characterized in that, The construction of semantic contrastive suggestion pairs based on training sample questions includes: Based on the training sample questions, construct positive and negative prompt statements; Based on the positive and negative prompts, the semantically opposing prompt pair is generated. The step of inputting the semantic opposition cue pair into a pre-trained large language model to obtain the semantic embedding vector corresponding to the training sample question includes: The positive and negative prompts are input into a pre-trained large language model to obtain the positive semantic embedding vector corresponding to the positive prompt and the negative semantic embedding vector corresponding to the negative prompt.

3. The large language model reasoning enhancement method according to claim 1, characterized in that, The control vector generator is constructed based on a lightweight neural network; The process involves training a control vector generator based on the sample problem control vector corresponding to the training sample problem, with the supervision objective of minimizing the deviation between the sample problem control vector and the semantic embedding vector, to obtain a trained control vector generator, including: The training sample questions are input into the text encoder to obtain the question embedding vectors corresponding to the training sample questions; The problem embedding vector is input into the control vector generator to obtain the sample problem control vector output by the control vector generator; Based on the deviation between the sample problem control vector and the semantic embedding vector, a mean squared error loss function is constructed. Based on the training sample problem and the mean squared error loss function, the control vector generator is trained to obtain the trained control vector generator.

4. The large language model reasoning enhancement method according to claim 3, characterized in that, The mean squared error loss function is specifically as follows: ； in, This represents the mean squared error loss value; Indicates the first Layer sample problem control vector; Indicates the first The layer's positive semantic embedding vector; Indicates the first Negative semantic embedding vector of the layer; This represents the total number of Transformer layers in the pre-trained large language model.

5. The large language model reasoning enhancement method according to claim 1, characterized in that, The step of obtaining the control weight factor corresponding to each layer in the trained control vector generator includes: Based on the thought chain data, the trained control vector generator is guided to learn and obtain the control weight factor corresponding to each layer.

6. The method for enhancing reasoning in a large language model according to claim 1, characterized in that, The control vector sequence is , where This represents the total number of Transformer layers in the pre-trained large language model; This represents the dimension of each layer of control vectors.

7. A large language model reasoning enhancement system, characterized in that, include: A semantic opposition prompt construction module is used to construct semantic opposition prompt pairs based on training sample questions, and input the semantic opposition prompt pairs into a pre-trained large language model to obtain the semantic embedding vectors corresponding to the training sample questions, wherein the semantic embedding vectors include positive semantic embedding vectors and negative semantic embedding vectors; The control vector generator training module is used to train the control vector generator based on the sample problem control vector corresponding to the training sample problem, with the goal of minimizing the deviation between the sample problem control vector and the semantic embedding vector. The module obtains the trained control vector generator and acquires the control weight factor corresponding to each layer in the trained control vector generator. The semantic embedding vector difference represents the difference between the positive semantic embedding vector and the negative semantic embedding vector. The control vector sequence generation module is used to input the problem to be processed into the trained control vector generator to obtain the control vector sequence output by the trained control vector generator, wherein the number of layers of the control vector sequence corresponds to the number of Transformer layers of the pre-trained large language model; The reasoning enhancement module is used to perform semantic activation on the Transformer module of the corresponding layer in the pre-trained large language model based on the control vector sequence and the control weight factor, so as to obtain the target reasoning result of the problem to be processed output by the pre-trained large language model. The reasoning enhancement module is specifically used for: Based on a preset control vector injection method, each layer of control vectors and the corresponding control weight factors in the control vector sequence are injected into the Transformer module of the corresponding layer in the pre-trained large language model for semantic activation. The preset control vector injection method includes layer-by-layer addition or multiplication fusion, and the injection position is the self-attention layer of the Transformer module.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the large language model reasoning enhancement method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the large language model reasoning enhancement method as described in any one of claims 1 to 6.

Images