Method and device for monitoring and controlling reasoning process based on large language model

Abstract

The invention provides an inference process monitoring and control method and device based on a large language model, and relates to the technical field of artificial intelligence. The method comprises the steps of obtaining a first hidden layer representation vector of each token in a reasoning sequence of a large language model, wherein the reasoning sequence is divided through an explicit reasoning mark of the large language model; calculating the relative position of each token in the reasoning sequence; regression modeling is carried out according to the first hidden layer representation vector and the relative position, a thinking progress vector is obtained, and the thinking progress vector represents the mapping relation between the hidden layer representation vector and the reasoning progress; in the reasoning process of the large language model, calculating the current reasoning progress according to the thinking progress vector and the second hidden layer representation vector of the current token; and according to the current reasoning progress, correcting the second hidden layer representation vector so as to adjust the reasoning path length of the large language model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a reasoning process monitoring and control method and device based on a large language model. BACKGROUND

[0002] Large language models (LLMs) have been widely used in mathematical calculations, logical reasoning, program generation, decision planning and other tasks due to their strong reasoning ability. Explicit structured reasoning, as an effective strategy, introduces a "thinking stage" separated by special markers before generating the final answer, enabling the model to generate intermediate chains of step-by-step reasoning, significantly improving the ability to handle complex problems.

[0003] However, the existing large language models have the following limitations in explicit reasoning: first, the current model cannot flexibly adjust the reasoning steps according to the task complexity, which may lead to under-reasoning or over-reasoning, affecting the correctness and generation efficiency of the answer; second, there is a lack of internal progress monitoring, and users cannot perceive the current reasoning progress of the model in real time, reducing the explainability and interactive experience; in addition, redundant reasoning steps increase the reasoning delay, and the energy consumption and computational cost during the reasoning process are significantly improved, especially when the reasoning depth is large. Therefore, how to monitor the reasoning process of the large language model and control the reasoning process is a technical problem to be solved. SUMMARY

[0004] In view of the above problems, the embodiments of the present application provide a reasoning process monitoring and control method and device based on a large language model, which can monitor the reasoning progress of the large language model and adjust the reasoning process of the large language model to reduce invalid reasoning steps and improve reasoning efficiency and correctness.

[0005] In a first aspect, the embodiments of the present application disclose a reasoning process monitoring and control method based on a large language model, comprising: obtaining a first hidden layer representation vector of each token in a reasoning sequence of a large language model, the reasoning sequence being divided by an explicit reasoning mark of the large language model; calculating the relative position of each token in the reasoning sequence, the relative position representing the reasoning progress of the large language model; performing regression modeling according to the first hidden layer representation vector and the relative position to obtain a thinking progress vector, the thinking progress vector representing the mapping relationship between the hidden layer representation vector and the reasoning progress; In the inference process of the large language model, a current inference progress is calculated according to the thinking progress vector and a second hidden layer representation vector of a current token; According to the current inference progress, the second hidden layer representation vector is corrected to adjust the inference path length of the large language model.

[0006] Optionally, obtaining the first hidden layer representation vector of each token in the inference sequence of the large language model comprises: Determining an inference task, the inference task being any one of: a mathematical calculation task, a logical inference task, a program generation task, and a decision planning task; Using the large language model to perform the inference task to obtain an inference sequence divided by the explicit inference mark; Taking the representation vector of each token in the last hidden layer of the inference sequence as the first hidden layer representation vector of each token.

[0007] Optionally, the explicit inference mark comprises a first explicit inference mark and a second explicit inference mark, and the inference sequence is a sequence generated between the first explicit inference mark and the second explicit inference mark.

[0008] Optionally, calculating the relative position of each token in the inference sequence comprises: According to the index of each token and the total length of the inference sequence, the relative position of each token in the inference sequence is calculated.

[0009] Optionally, the thinking progress vector is obtained by regression modeling according to the first hidden layer representation vector and the relative position, comprising: Taking the first hidden layer representation vector and the relative position as training samples, the hidden layer representation vector is input into a regression model to obtain a predicted relative position; The regression model is updated to minimize the mean square error of the relative position and the predicted relative position, and the thinking progress vector is obtained.

[0010] Optionally, in the inference process of the large language model, a current inference progress is calculated according to the thinking progress vector and a second hidden layer representation vector of a current token, comprising: In the inference process of the large language model, a mapping value of the second hidden layer representation vector on the thinking progress vector is calculated to obtain the current inference progress.

[0011] Optionally, the method further comprises: Converting the current inference progress into a progress percentage; A graphical interface is drawn according to the progress percentage, and the graphical interface is displayed, the graphical interface being any one of the following: a progress bar, a scroll bar, and a ring-shaped progress chart.

[0012] Optionally, according to the current inference progress, the second hidden layer representation vector is corrected to adjust an inference path length of the large language model, including: According to the current inference progress, a control parameter is determined, the control parameter being used to accelerate or delay an inference process; According to the control parameter and the thought progress vector, the second hidden layer representation vector is corrected to obtain a corrected hidden layer representation vector; The corrected hidden layer representation vector is input into the large language model for inference.

[0013] Optionally, the method further includes: In a case where the current inference progress is equal to a progress threshold or the large language model generates the second explicit inference mark, the large language model is controlled to end inference and output an inference result.

[0014] A second aspect of the embodiments of the present application discloses a large language model-based inference process monitoring and control device, the device including: A vector acquisition module is configured to acquire a first hidden layer representation vector of each token in an inference sequence of a large language model, the inference sequence being divided by an explicit inference mark of the large language model; A position calculation module is configured to calculate a relative position of each token in the inference sequence, the relative position representing an inference progress of the large language model; A thought modeling module is configured to perform regression modeling according to the first hidden layer representation vector and the relative position to obtain a thought progress vector, the thought progress vector representing a mapping relationship between the hidden layer representation vector and the inference progress; A progress monitoring module is configured to calculate a current inference progress according to the thought progress vector and a second hidden layer representation vector of a current token in an inference process of the large language model; A progress control module is configured to correct the second hidden layer representation vector according to the current inference progress to adjust an inference path length of the large language model.

[0015] A third aspect of the embodiments of the present application discloses an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the large language model-based inference process monitoring and control method of the first aspect of the embodiments of the present application when executing the computer program.

[0016] In a fourth aspect, the embodiments of the present application disclose a readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the inference process monitoring and control method based on a large language model according to the first aspect of the embodiments of the present application.

[0017] In a fifth aspect, the embodiments of the present application disclose a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the inference process monitoring and control method based on a large language model according to the first aspect of the embodiments of the present application.

[0018] The embodiments of the present application have the following advantages: In the embodiments of the present application, by obtaining the first hidden layer representation vector of each token in the inference sequence of the large language model, and combining the relative positions of the tokens in the inference sequence to perform regression modeling, the abstract "thinking stage" of the large language model that is originally invisible is converted into a quantifiable "thinking progress vector", which represents the mapping relationship between the hidden layer representation vector and the inference progress. Therefore, during the inference process of the large language model, the current inference progress can be calculated according to the thinking progress vector, and the inference progress of the large language model is monitored to accurately grasp the progress of the model in the inference task (such as mathematical calculation, logical inference, etc.).

[0019] Moreover, according to the calculated current inference progress, the second hidden layer representation vector of the large language model is dynamically corrected, so as to actively intervene and adjust the inference path of the large language model. Such intervention is performed during the model calculation process, which can effectively guide the large language model to avoid falling into a circular inference, irrelevant divergence or known error path, and ensure that the inference process efficiently progresses towards the correct direction of solving the problem. In this way, the embodiments of the present application can monitor the inference progress of the large language model and adjust the inference path length of the large language model, thereby reducing the invalid inference steps and improving the inference efficiency and correctness of the model in complex tasks. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative labor.

[0021] Figure 1 is a step flowchart of the inference process monitoring and control method based on a large language model provided by the embodiments of the present application; Figure 2is a step flowchart of another large language model-based reasoning process monitoring and control method provided by an embodiment of the present application. Figure 3 is a structural schematic diagram of a large language model-based reasoning process monitoring and control device provided by an embodiment of the present application. Figure 4 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0022] To make the above objectives, characteristics and advantages of the present application more apparent, clear and complete, the technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0023] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship.

[0024] In order to better understand the technical solutions of the present application, a brief description of the related art is first given.

[0025] As an effective strategy, explicit reasoning enables the model to generate intermediate chains of step-by-step reasoning by introducing "thinking stages" separated by special markers before generating the final answer, significantly improving the ability to handle complex problems. However, existing large language models have the following problems in explicit reasoning: Problem 1: Uncontrollable length of thinking path. The length of the "thinking stage" of model reasoning has a great impact on the result. If the reasoning is too short, it may not cover the complexity of the task, resulting in underthinking; if the reasoning is too long, it may fall into redundancy, repeated verification, or even invalid loops, forming overthinking, which not only increases the computational cost, but also may reduce the accuracy of the answer. Existing methods mostly affect the reasoning length through prompt engineering or training stage constraints, but lack a direct and controllable mechanism.

[0026] Problem 2: Lack of internal progress monitoring mechanism. Users or the system cannot know in real time the model's "thinking progress (reasoning progress)" from the final answer during the reasoning process. This leads to insufficient transparency in the reasoning process, and users cannot determine whether the model has gotten stuck in redundant reasoning or is close to convergence, which severely restricts the model's usability in high-risk or time-sensitive tasks.

[0027] Question 3: Limitations of existing control methods. Related techniques indirectly control inference depth through training optimization (such as Reinforcement Learning with Length Control) or dynamically adjusting cues during inference. However, these methods essentially remain at the level of input or external constraints and cannot directly affect the internal representation of the model, lacking accuracy and real-time performance.

[0028] Question 4: Overcomputation and Energy Consumption. In resource-constrained environments, excessively long inference chains significantly increase inference time and energy consumption, reducing the practical deployment value of the model. This problem is particularly pronounced in mobile devices, embedded devices, or large-scale distributed inference scenarios.

[0029] Based on the above problems, there is an urgent need for a new method that can monitor the reasoning progress in real time and dynamically intervene in the length of the reasoning path, so as to reduce invalid calculations, alleviate over-reasoning, and improve reasoning efficiency and accuracy.

[0030] To overcome the limitations of related problems, this application provides a method for monitoring and controlling the reasoning process based on a large language model. This method introduces a Thinking Progress Vector (TPV) to achieve precise monitoring and intervention of the reasoning process of a large language model.

[0031] The following detailed description, in conjunction with the accompanying drawings, of the inference process monitoring and control method based on a large language model provided in this application, through specific embodiments and application scenarios, will be provided in detail.

[0032] Reference Figure 1 As shown, Figure 1 This is a flowchart illustrating the steps of a method for monitoring and controlling the reasoning process based on a large language model, as provided in an embodiment of this application. Figure 1 As shown, the inference process monitoring and control method based on a large language model may include steps S110 to S150: Step S110: Obtain the first hidden layer representation vector of each token in the inference sequence of the large language model, wherein the inference sequence is divided by the explicit inference tags of the large language model.

[0033] The large language model refers to a large language model supporting explicit reasoning, that is, the large language model is a model that divides the reasoning stage and the reasoning result by using an explicit reasoning mark. For example, the large language model can be DeepSeek-R1-Distill-Qwen-32B, DeepSeek-R1-Distill-LLaMA-8B, and the like. These models generate a reasoning sequence divided by the explicit reasoning mark in the reasoning process.

[0034] In addition, the large language model can be used to perform mathematical calculation tasks, logical reasoning tasks, program generation tasks, decision planning tasks, and the like. In the reasoning process, the large language model generates a reasoning sequence (reasoning path), and the reasoning sequence includes a plurality of tokens (tokens). The first hidden layer representation vector of each token is extracted in turn.

[0035] It can be understood that the first hidden layer representation vector can be a representation vector of a specific hidden layer (such as the second last layer, the last layer, and the like) of the large language model, or a representation vector of a plurality of hidden layers, which is not limited in the embodiments of the present application.

[0036] Step S120: calculating the relative position of each token in the reasoning sequence, the relative position representing the reasoning progress of the large language model.

[0037] In the embodiments of the present application, the plurality of tokens in the reasoning sequence are arranged in the order of generation time, for example, the first token in the reasoning sequence is the first token generated in the reasoning process, and the last token in the reasoning sequence is the last token generated in the reasoning process. Therefore, the relative position of each token in the reasoning sequence can accurately represent the reasoning progress of the large language model.

[0038] Step S130: regression modeling according to the first hidden layer representation vector and the relative position to obtain a thinking progress vector, the thinking progress vector representing the mapping relationship between the hidden layer representation vector and the reasoning progress.

[0039] In the embodiments of the present application, the first hidden layer representation vector and the relative position of each token are taken as a training data, and a data set is obtained according to a plurality of training data. The data set contains the mapping relationship between the hidden layer representation vector and the reasoning progress.

[0040] Using a linear regression method, regression modeling is performed on each training data (the first hidden layer representation vector and the relative position) in the training data set, so as to obtain the mapping relationship between the hidden layer representation vector and the reasoning progress, that is, the thinking progress vector.

[0041] Step S140: In the inference process of the large language model, the current inference progress is calculated according to the thinking progress vector and the second hidden layer representation vector of the current token.

[0042] In the embodiment of the present application, after obtaining the thinking progress vector through the method of steps S110 to S130, the thinking progress vector can be used to monitor the inference process of the large language model when performing a new inference task. Specifically, in the inference process of the large language model, the second hidden layer representation vector of the current token is obtained. Since the thinking progress vector represents the mapping relationship between the hidden layer representation vector and the inference progress, the current inference progress can be calculated according to the thinking progress vector and the second hidden layer representation vector of the current token.

[0043] The current inference progress can be understood as the current thinking progress of the large language model, and the current inference progress of the large language model can be perceived according to the current inference progress.

[0044] It can be understood that the second hidden layer representation vector and the first hidden layer representation vector come from the same hidden layer to ensure the consistency of the feature space. For example, if the first hidden layer representation vector is the representation vector of the last hidden layer, the second hidden layer representation vector is also the representation vector of the last hidden layer; if the first hidden layer representation vector is the representation vector of the second last hidden layer, the second hidden layer representation vector is also the representation vector of the second last hidden layer.

[0045] In this way, the thinking progress of the large language model is concretized into a calculable index (inference progress) through the thinking progress vector, thereby enhancing the transparency and interpretability of the large language model when performing inference tasks. Users can understand and diagnose the internal state of the model according to the "current inference progress", which provides key technical support for debugging the model, establishing trust, and applying in high-risk fields (such as medical care and finance), and improves the reliability and controllability of the model behavior.

[0046] Step S150: According to the current inference progress, the second hidden layer representation vector is corrected to adjust the inference path length of the large language model.

[0047] In the inference process of the large language model, the inference path length of the large language model can also be adjusted according to the current inference progress. For example, if the current inference progress is too fast, the second hidden layer representation vector is corrected to slow down the inference progress (process); if the current inference progress is too slow, the second hidden layer representation vector is corrected to speed up the inference progress (process).

[0048] By extracting and analyzing the first hidden layer representation vectors of each token in the inference sequence inside the large language model, and combining the relative positions of each token in the inference sequence to perform regression modeling, the abstract "thinking stage" of the large language model that is originally invisible is converted into a quantifiable "thinking progress vector". According to the thinking progress vector, the inference progress of the large language model can be monitored to accurately grasp the progress of the model in the inference task. Moreover, according to the calculated current inference progress, the second hidden layer representation vector of the large language model is dynamically corrected, so as to actively intervene and adjust the inference path of the large language model; and such intervention is performed during the model calculation process, which can effectively guide the large language model to avoid falling into a circular inference, irrelevant divergence or known error path, and ensure that the inference process efficiently progresses towards the correct direction of solving the problem.

[0049] Therefore, the embodiment of the present application can monitor the inference progress of the large language model and adjust the inference path length of the large language model, thereby reducing invalid inference steps and improving the inference efficiency and correctness of the model in complex tasks. It is worth noting that the present application is based on the inherent hidden layer representation and inference sequence structure of the model, and does not rely on prior knowledge of specific model architectures or tasks. Therefore, the method can be widely applied to various large language models and is suitable for any task scenario that requires multi-step inference, such as question answering, code generation, etc., and has good universality and expandability.

[0050] In an optional embodiment, the step S110 of "obtaining the first hidden layer representation vector of each token in the inference sequence of the large language model" can specifically include the following steps S110-1 to S110-3: Step S110-1: determining an inference task, wherein the inference task is any one of the following: a mathematical calculation task, a logical inference task, a program generation task, and a decision planning task; Step S110-2: performing the inference task using the large language model to obtain an inference sequence divided by the explicit inference mark; Step S110-3: taking the representation vector of each token in the last hidden layer of the inference sequence as the first hidden layer representation vector of each token.

[0051] In the embodiment of the present application, the inference task is determined to ensure that the large language model can generate a complete inference sequence in the inference stage. A random inference task can be selected from various inference tasks to be executed by the large language model to obtain an inference sequence divided by an explicit inference mark.

[0052] Optionally, the explicit inference markers include a first explicit inference marker and a second explicit inference marker, and the inference sequence is a sequence generated between the first explicit inference marker and the second explicit inference marker. For example, the first explicit inference marker is <think>, the second explicit inference label is< / think> , and all generated sequences between <think>to< / think> are intercepted as inference sequences.

[0053] All tokens in the inference sequence are collected, and for each token, a representation vector of a last layer hidden layer thereof is extracted as a first hidden layer representation vector of the token. For example, the first hidden layer representation vector of the jth token can be represented as , wherein d represents a feature dimension of the first hidden layer representation vector.

[0054] By adopting the technical solutions of the embodiments of the present application, the inference sequence and the first hidden layer representation vector of each token can be obtained from a specified complex inference task, thereby providing accurate and high-quality data input for subsequent calculation of relative positions and regression modeling to obtain a "thinking progress vector".

[0055] In an optional embodiment, the step S120 of "calculating the relative positions of the tokens in the inference sequence" specifically includes: calculating the relative positions of the tokens in the inference sequence according to the indexes of the tokens and the total length of the inference sequence.

[0056] wherein the total length of the inference sequence can be understood as the total number of tokens in the inference sequence, and the index of the token can be understood as the absolute position of the token. According to the index of the token and the total length of the inference sequence, the relative position of the token in the inference sequence is calculated, which is a normalized value for representing the progress stage of the token in the inference sequence, i.e., the inference progress of the large language model.

[0057] For example, the relative position of the jth token in the inference sequence can be represented as:

[0058] wherein j represents the index of the token, and N represents the total length of the inference sequence. Through the above formula, the relative position of the token in the inference sequence can be mapped to the interval [0, 1], i.e., the value of the relative position is a value between 0 and 1.

[0059] By adopting the technical solutions of the embodiments of the present application, the absolute position (index) of each token in the reasoning sequence can be converted into a standardized relative position value that can effectively represent the reasoning progress of the token in the sequence, and the relative position combined with the first hidden layer representation vector of the corresponding token constitutes a basic data pair for establishing a "hidden layer representation-reasoning progress" mapping relationship.

[0060] In an optional embodiment, the step S130 of "performing regression modeling on the first hidden layer representation vector and the relative position to obtain a thinking progress vector" can specifically include steps S1301 to S130-2: Step S130-1: taking the first hidden layer representation vector and the relative position as training samples, inputting the hidden layer representation vector into a regression model to obtain a predicted relative position; Step S130-2: updating the regression model to minimize the mean square error of the relative position and the predicted relative position to obtain the thinking progress vector.

[0061] In the embodiments of the present application, the first hidden layer representation vector and the relative position are taken as training samples to train a regression model (for example, the regression model can be a single-layer linear layer or a shallow neural network). Specifically, for each training sample, the first hidden layer representation vector is taken as input and the corresponding relative position is taken as a training target for regression training, and the training target is to enable the regression model to learn to predict the relative position (i.e., reasoning progress) of a token in the reasoning sequence of a large language model from the hidden layer representation of the token.

[0062] For example, the regression model can be represented as:

[0063] wherein, is a parameter of the regression model, i.e., a thinking progress vector, and h represents a hidden layer representation vector.

[0064] The target of minimizing the mean square error of the relative position and the predicted relative position can be represented as:

[0065] wherein, is a predicted relative position, is the mean square error of the relative position and the predicted relative position.

[0066] The regression model is trained according to the method of steps S130-1 to S130-2, and when a condition for ending the training is met (for example, the accuracy of the regression model converges to a preset accuracy, or the number of training times meets a preset number, etc.), a trained regression model is obtained. The trained regression model has learned the ability to accurately predict the corresponding reasoning progress from the hidden layer representation vector, and the parameters of the trained regression model are taken as the thinking progress vector.

[0067] Optionally, the step S130 further includes the following step: smoothing the predicted relative position by using a sequence model (such as RNN or GRN) to obtain a smoothed predicted relative position.

[0068] Further, the step S130-2 is specifically: updating the regression model to minimize the mean square error of the smoothed predicted relative position and the predicted relative position, to obtain the thinking progress vector. In this way, the robustness of the regression model is enhanced, and the accuracy of the thinking progress vector is ensured.

[0069] By adopting the technical solutions of the embodiments of the present application, a regression model is trained according to the first hidden layer representation vector and the relative position to obtain a thinking progress vector, which encapsulates a stable and universal mapping relationship between the internal hidden layer state and the reasoning progress of the large language model during the reasoning process. In subsequent real-time reasoning control, the thinking progress vector is used to calculate the current reasoning progress, thereby realizing the monitoring and guidance of the model reasoning process.

[0070] In an optional embodiment, the step S140 of "in the reasoning process of the large language model, calculating the current reasoning progress according to the thinking progress vector and the second hidden layer representation vector of the current token" can specifically include: in the reasoning process of the large language model, calculating the mapping value of the second hidden layer representation vector on the thinking progress vector to obtain the current reasoning progress.

[0071] In the embodiments of the present application, the thinking progress vector represents the mapping relationship between the hidden layer representation vector and the reasoning progress, and the second hidden layer representation vector is mapped according to the thinking progress vector to obtain the current reasoning progress. For example, the current reasoning progress may be expressed as:

[0072] It can be understood that the current reasoning progress represents the reasoning progress of the large language model at the current time, and the current reasoning progress is a value between 0 and 1. The closer the value of the current reasoning progress to 0, the more the reasoning is in the initial stage. The closer the value of the current reasoning progress to 1, the closer the reasoning is to completion.

[0073] In an alternative embodiment, after step S140, the following steps can also be included: converting the current reasoning progress into a progress percentage; drawing a graphical interface according to the progress percentage, and displaying the graphical interface, the graphical interface being any one of the following: a progress bar, a scroll bar, a ring-shaped progress chart.

[0074] In the embodiments of the present application, the current reasoning progress can also be visually displayed, and the current reasoning progress is converted into a progress percentage, which is a percentage between 0% and 100%. The progress percentage is dynamically mapped to a graphical interface, and the current thinking progress of the large language model is displayed to the user.

[0075] The progress bar draws a horizontal (vertical) bar in the interface, and the proportion of the filled area dynamically increases with the change of the progress percentage, from 0% to 100%. The filled part can be coded using color (for example, blue represents normal, yellow represents attention, and red represents possible lag). The ring-shaped progress chart draws a ring in the interface, and the arc length of the ring is equal to the progress percentage. This form is compact and beautiful, and is suitable for embedding in the corner of the user interface. The scroll bar simulates the progress bar of a video player, providing a slider that can visually display the current position. The position of the slider is determined according to the progress percentage, which can effectively convey the information of "where the reasoning has proceeded to".

[0076] In this way, the abstract current reasoning is converted into a visual graphical interface, enabling the user to intuitively perceive the "thinking" process (reasoning progress) of the large language model.

[0077] In an alternative embodiment, step S150 "according to the current reasoning progress, correcting the second hidden layer representation vector to adjust the reasoning path length of the large language model" can specifically include sub-steps S150-1 to S150-3: Step S150-1: determining a control parameter according to the current reasoning progress, the control parameter being used to accelerate the reasoning process or to delay the reasoning process; Step S150-2: correcting the second hidden layer representation vector according to the control parameter and the thinking progress vector to obtain a corrected hidden layer representation vector; Step S150-3: inputting the corrected hidden layer representation vector into the large language model for reasoning.

[0078] In the embodiments of the present application, the control parameter determined according to the current reasoning progress is a scalar, and its sign and size determine the direction and intensity of the correction. Specifically, if the current reasoning progress grows too slowly or is lower than the expected progress curve within a period of time, it indicates that the model may be stuck in the current train of thought, and a positive control parameter is generated, that is, Overclocking is implemented to speed up the reasoning process and shorten the reasoning path length; if the current reasoning progress grows abnormally fast, it indicates that the model may not think deeply and carefully enough, and a negative control parameter is generated naturally, that is, Downclocking is implemented to slow down the reasoning process and lengthen the reasoning path length.

[0079] According to the control parameter and the thought progress vector, the second hidden layer representation vector is corrected. The core idea of the correction is to apply a perturbation to the current hidden state (i.e., the second hidden layer representation vector) in the direction of the "thought progress" (i.e., the direction defined by the thought progress vector), thereby guiding the model's thought state to shift forward or backward.

[0080] For example, the corrected hidden layer representation vector may be represented as:

[0081] wherein, represents the thought progress vector, represents the second hidden layer representation vector; represents the control parameter, that is, represents speeding up the reasoning progress, and the model tends to use fewer tokens to complete the current reasoning step, thereby shortening the overall reasoning path length and improving the efficiency; represents slowing down the reasoning progress, and the model will be encouraged to generate more rich intermediate content at the current step, thereby lengthening the overall reasoning path length and increasing the rigor and accuracy of reasoning.

[0082] The corrected hidden layer representation vector is input into the large language model for reasoning to generate the next token. As the reasoning steps proceed, the reasoning progress is overall accelerated or slowed down, achieving dynamic adjustment of the reasoning path length.

[0083] The technical solution of the embodiments of the present application realizes control of the reasoning process of the large language model, can dynamically optimize the reasoning path, and effectively improves the reasoning efficiency while ensuring the reliability of the result.

[0084] In an optional embodiment, after the step S140, the method further includes the following steps: If the current inference progress equals the progress threshold or the large language model generates the second explicit inference tag, the large language model is controlled to end the inference and the inference result is output.

[0085] In this embodiment, the progress threshold can be 1. If the current inference progress equals the progress threshold, it indicates that the model's internal thinking state has reached the "near completion" stage, and the state of its hidden layer representation vectors sufficiently indicates that the inference is approaching completion. At this point, the large language model can end the inference. If the large language model generates a second explicit inference marker, it indicates that the model itself has explicitly issued a signal of "inference completed, conclusion to be given" through its structured output. At this point, the large language model can end the inference.

[0086] After the large language model finishes inference, all generated results up to the termination time are output as the final inference result.

[0087] The technical solution adopted in this application realizes intelligent judgment of the end node of the reasoning process. It combines the dual information of the model's internal state (current reasoning progress) and external performance (generated explicit reasoning markers), making the termination of the reasoning process more accurate and efficient. It not only prevents the model from continuing to generate irrelevant content after the conclusion is clear, thus avoiding the waste of resources, but also ensures the integrity and readability of the final output results.

[0088] The following describes the inference process monitoring and control method based on a large language model provided in this application, using a specific embodiment as an example. (Refer to...) Figure 2 As shown, Figure 2 This is a flowchart of another method for monitoring and controlling the inference process based on a large language model provided in this application embodiment. The method includes steps S210 to S270: Step S210: Determine the reasoning task, which is any of the following: mathematical calculation task, logical reasoning task, program generation task, decision planning task; execute the reasoning task using the large language model to obtain a reasoning sequence divided by the explicit reasoning tags; take the representation vector of each token in the last hidden layer in the reasoning sequence as the first hidden layer representation vector of each token.

[0089] Step S220: Calculate the relative position of each token in the inference sequence based on the index of each token and the total length of the inference sequence.

[0090] Step S230: input the first hidden layer representation vector and the relative position as a training sample into a regression model to obtain a predicted relative position; update the regression model by minimizing the mean square error of the relative position and the predicted relative position, and obtain the thinking progress vector.

[0091] Step S240: in the inference process of the large language model, calculate the mapping value of the second hidden layer representation vector on the thinking progress vector to obtain the current inference progress.

[0092] Step S250: convert the current inference progress into a progress percentage; draw a graphical interface according to the progress percentage, and display the graphical interface, wherein the graphical interface is any one of a progress bar, a scroll bar, and a ring-shaped progress chart.

[0093] Step S260: determine a control parameter according to the current inference progress, wherein the control parameter is used to accelerate or delay the inference process; modify the second hidden layer representation vector according to the control parameter and the thinking progress vector to obtain a modified hidden layer representation vector; and input the modified hidden layer representation vector into the large language model for inference.

[0094] Step S270: in the case where the current inference progress is equal to a progress threshold or the large language model generates the second explicit inference mark, control the large language model to end the inference and output an inference result.

[0095] In the embodiments of the present application, the current inference progress is displayed to the user through a progress bar and other visual methods to show the inference progress, thereby enhancing the model interpretability and facilitating the user to understand and debug the model inference process. The thinking progress vector is used to directly intervene in the model inference path length, thereby breaking through the limitation of traditional dependence on prompt engineering or training optimization, accurately controlling the inference depth, effectively reducing the redundant inference steps, maintaining or improving the answer correctness, and significantly reducing the computing overhead and inference delay. In this way, the method can significantly improve the inference accuracy under the same computing budget, and the performance improvement is more significant under low computing budget. Moreover, the model structure does not need to be modified, and only the hidden state in the inference stage needs to be accessed to realize the monitoring and intervention, thereby having good universality and scalability.

[0096] The method provided by the embodiments of the present application can be extended to various application scenarios. Specifically, in a mathematical and logical reasoning task, an over-verification cycle can be avoided, and the answering speed and accuracy can be improved; in a natural language reasoning and question answering system, reasoning transparency and interactive controllability can be provided for users; in a resource-limited environment (such as a mobile device or an embedded system), by reducing redundant reasoning steps, the calculation consumption and energy consumption can be reduced; in a scenario requiring rapid decision-making (such as a dialogue system or a search question and answer), the timeliness of the response can be ensured.

[0097] The embodiments of the present application also provide a reasoning process monitoring and control device based on a large language model, as shown in Figure 3 Figure 3 FIG. 1 is a structural schematic diagram of a reasoning process monitoring and control device based on a large language model provided by the embodiments of the present application. The device comprises: The obtaining module 310 is configured to obtain a first hidden layer representation vector of each token in a reasoning sequence of a large language model, wherein the reasoning sequence is divided by an explicit reasoning mark of the large language model. The calculation module 320 is configured to calculate a relative position of each token in the reasoning sequence, wherein the relative position represents a reasoning progress of the large language model. The modeling module 330 is configured to perform regression modeling according to the first hidden layer representation vector and the relative position, to obtain a thinking progress vector, wherein the thinking progress vector represents a mapping relationship between the hidden layer representation vector and the reasoning progress. The monitoring module 340 is configured to calculate a current reasoning progress according to the thinking progress vector and a second hidden layer representation vector of a current token in a reasoning process of the large language model. The control module 350 is configured to correct the second hidden layer representation vector according to the current reasoning progress, to adjust a reasoning path length of the large language model.

[0098] In an optional embodiment, the obtaining module is further configured to: determine a reasoning task, wherein the reasoning task is any one of a mathematical calculation task, a logical reasoning task, a program generation task, and a decision planning task; execute the reasoning task by using the large language model, to obtain a reasoning sequence divided by the explicit reasoning mark; take a representation vector of each token in the last hidden layer of the reasoning sequence as the first hidden layer representation vector of each token.

[0099] ​In an optional embodiment, the explicit inference mark includes a first explicit inference mark and a second explicit inference mark, and the inference sequence is a sequence generated between the first explicit inference mark and the second explicit inference mark.

[0100] In an optional embodiment, the computing module is further configured to calculate a relative position of each token in the inference sequence according to an index of each token and a total length of the inference sequence.

[0101] In an optional embodiment, the modeling module is further configured to: input the hidden layer representation vector into a regression model to obtain a predicted relative position, with the first hidden layer representation vector and the relative position as training samples; update the regression model to obtain the thinking progress vector, with a goal of minimizing a mean square error of the relative position and the predicted relative position.

[0102] In an optional embodiment, the control module is further configured to calculate a mapping value of the second hidden layer representation vector on the thinking progress vector to obtain the current inference progress during an inference process of the large language model.

[0103] In an optional embodiment, the device further includes: a display module configured to convert the current inference progress into a progress percentage, draw a graphical interface according to the progress percentage, and display the graphical interface, the graphical interface being any one of a progress bar, a scroll bar, and a ring-shaped progress chart.

[0104] In an optional embodiment, the control module is further configured to: determine a control parameter according to the current inference progress, the control parameter being used to accelerate an inference process or to delay the inference process; correct the second hidden layer representation vector to obtain a corrected hidden layer representation vector according to the control parameter and the thinking progress vector; and input the corrected hidden layer representation vector into the large language model for inference.

[0105] In an optional embodiment, the device further includes: an output module configured to control the large language model to end inference and output an inference result in a case where the current inference progress is equal to a progress threshold or the large language model generates the second explicit inference mark.

[0106] It can be understood that the inference process monitoring and control device based on the large language model in the embodiments of the present application can implement the inference process monitoring and control method based on the large language model in the above embodiments. The inference process monitoring and control device based on the large language model has the same advantages as the inference process monitoring and control method based on the large language model in the above embodiments, and will not be described here.

[0107] The embodiments of the present application also provide an electronic device, which refers to Figure 4 , Figure 4 is a structural schematic diagram of an electronic device provided by the embodiments of the present application. As shown in Figure 4 , the electronic device 400 includes a memory 410 and a processor 420, the memory 410 and the processor 420 are communicatively connected through a bus, and the memory 410 stores a computer program, which can run on the processor 420, and further implement the steps of the inference process monitoring and control method based on the large language model described in the embodiments of the present application.

[0108] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the inference process monitoring and control method based on the large language model described in the embodiments of the present application.

[0109] The embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps of the inference process monitoring and control method based on the large language model described in the embodiments of the present application.

[0110] Each of the embodiments in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts of each embodiment can be referred to each other.

[0111] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general purpose computer, a special purpose computer, an embedded processor, or other programmable data processing terminal equipment to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal equipment produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0112] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0114] Although preferred embodiments of the application have been described, those skilled in the art will recognize that additional modifications and changes can be made thereto without departing from the scope of the application. Accordingly, the appended claims are intended to cover all such modifications and changes as fall within the scope of the application.

[0115] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are more especially used for the purpose of distinction from other elements in the specification. Also, the terms "include", "comprise", or their other variants are intended to encompass non-exclusive inclusions, such that processes, methods, articles, or apparatuses that comprise a list of elements are not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such processes, methods, articles, or apparatuses. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose additional identical elements of the process, method, article, or apparatus including the specific element. Nothing is stated in the above specification that can not be otherwise implied by the terms as used in the specification.

[0116] The above provides a large language model-based reasoning process monitoring and control method and device, and the principles and implementation modes of the present application are described in the text by applying specific examples. The above description of the embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for monitoring and controlling inference processes based on large language models, characterized in that, The method comprises: obtaining a first hidden layer representation vector of each token in an inference sequence of a large language model, the inference sequence being divided by an explicit inference mark of the large language model; calculating a relative position of each token in the inference sequence, the relative position representing an inference progress of the large language model; performing regression modeling according to the first hidden layer representation vector and the relative position to obtain a thinking progress vector, the thinking progress vector representing a mapping relationship between the hidden layer representation vector and the inference progress; in an inference process of the large language model, calculating a current inference progress according to the thinking progress vector and a second hidden layer representation vector of a current token; correcting the second hidden layer representation vector according to the current inference progress to adjust a length of an inference path of the large language model.

2. The method of claim 1, wherein, The method comprises: determining an inference task, the inference task being any one of a mathematical calculation task, a logical inference task, a program generation task, and a decision planning task; executing the inference task by using the large language model to obtain an inference sequence divided by the explicit inference mark; taking a representation vector of each token in a last hidden layer of the inference sequence as a first hidden layer representation vector of each token.

3. The method according to claim 1 or 2, characterized in that, The explicit inference mark comprises a first explicit inference mark and a second explicit inference mark, and the inference sequence is a sequence generated between the first explicit inference mark and the second explicit inference mark.

4. The method of claim 1, wherein, The method comprises: calculating the relative position of each token in the inference sequence according to an index of each token and a total length of the inference sequence.

5. The method of claim 1, wherein, The method comprises: taking the first hidden layer representation vector and the relative position as training samples, inputting the hidden layer representation vector into a regression model to obtain a predicted relative position; updating the regression model to obtain the thinking progress vector, with a minimum mean square error of the relative position and the predicted relative position as a target.

6. The method of claim 1, wherein, The method comprises: in the inference process of the large language model, calculating a mapping value of the second hidden layer representation vector on the thinking progress vector to obtain the current inference progress.

7. The method according to claim 1 or 6, characterized in that, The method further comprises: converting the current inference progress into a progress percentage; drawing a graphical interface according to the progress percentage, and displaying the graphical interface, the graphical interface being any one of a progress bar, a scroll bar, and a ring-shaped progress diagram.

8. The method of claim 1, wherein, The method comprises: According to the current inference progress, a control parameter is determined, the control parameter being used to accelerate the inference process or to delay the inference process; According to the control parameter and the thinking progress vector, the second hidden layer representation vector is revised to obtain a revised hidden layer representation vector; The revised hidden layer representation vector is input into the large language model for inference.

9. The method of claim 3, wherein, The method further comprises: In a case where the current inference progress is equal to a progress threshold or the large language model generates the second explicit inference mark, the large language model is controlled to end the inference, and an inference result is output.

10. A large language model based inference process monitoring and control apparatus, characterized by, Comprise: An acquisition module is configured to acquire a first hidden layer representation vector of each token in an inference sequence of a large language model, the inference sequence being divided by an explicit inference mark of the large language model; A calculation module is configured to calculate a relative position of each token in the inference sequence, the relative position representing an inference progress of the large language model; A modeling module is configured to perform regression modeling according to the first hidden layer representation vector and the relative position to obtain a thinking progress vector, the thinking progress vector representing a mapping relationship between the hidden layer representation vector and the inference progress; A monitoring module is configured to calculate a current inference progress according to the thinking progress vector and a second hidden layer representation vector of a current token in an inference process of the large language model; A control module is configured to revise the second hidden layer representation vector according to the current inference progress to adjust a length of an inference path of the large language model.

Images