An uncertainty-aware trusted large model inference time alignment method and device

Abstract

The application discloses an uncertainty-aware trusted large model inference alignment method and device, and relates to the technical field of natural language processing. The method comprises the following steps: acquiring a natural language prompt text input by a user, performing forward inference on the natural language prompt text by using a large language model with frozen parameters, acquiring candidate word information at a current time step, performing temperature scaling calibration on original output of a lightweight alignment model by using a temperature coefficient, obtaining a calibrated probability distribution and calculating a cognitive confidence; obtaining all information surprise degrees of a current word by using corpus statistical data constructed in advance, and performing normalization processing to obtain semantic saliency of the current candidate word; calculating a joint necessity score and a current dynamic threshold based on the cognitive confidence and the semantic saliency of the current candidate word; comparing the current dynamic threshold with the joint necessity score, outputting a comparison result, and generating a final text according to the comparison result. The application can improve the answer accuracy.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to an alignment method and apparatus for reasoning with a reliable large model based on uncertainty perception. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models (LMs) have demonstrated remarkable capabilities in natural language generation, reasoning, and medical question-answering systems. However, ensuring that the output of these models aligns with human values ​​and preventing the generation of harmful, biased, or false information has become a core challenge in this field. Current alignment techniques are mainly divided into two categories: training-time alignment methods and inference-time alignment methods.

[0003] Representative methods for training-time alignment include reinforcement learning based on human feedback and direct preference optimization. While effective, these methods require collecting expensive human preference data and updating massive amounts of model parameters. As the model size continues to increase, this retraining or fine-tuning of all parameters incurs extremely high computational costs and resource consumption. To address the training cost issue, inference-time alignment has emerged as an emerging paradigm. This paradigm does not change the parameters of a large language model with frozen parameters, but instead guides the direction of generation by introducing external control signals during the model's text generation process. This approach treats alignment as a dynamic control problem, offering advantages such as modularity, pluggability, and the ability to avoid retraining the model.

[0004] In the field of secure alignment of large language models, to avoid the high computational cost and catastrophic forgetting risk of full-parameter retraining, the industry has begun to adopt "inference-time alignment" as a more efficient and lightweight solution. Currently, this technical approach has mainly evolved into three typical implementation paths. Although their mathematical forms are different, they all adopt a dense control strategy of "word-by-word intervention".

[0005] The first category is supervised schemes based on micro-agents, with a typical example being Microtoken-level Accept-Reject Alignment (MARA). This scheme follows a "weak-to-strong" guiding principle, introducing a lightweight micro-agent model with fewer parameters outside the large base model with frozen parameters. Its specific working principle is to establish a closed loop of "generation-verification-correction": during inference, each time the large language model with frozen parameters generates a candidate token or its corresponding hidden state, the micro-agent model uses it as input to perform a binary classification task or logical judgment. The micro-agent calculates the probability that the current output conforms to the security specifications and determines whether it is "accepted" or "rejected". Once it is determined to be potentially insecure (i.e., "rejected"), the system immediately interrupts the current generation path and forces the model to regenerate compliant tokens through resampling or backtracking mechanisms.

[0006] The second category is based on contrastive decoding, with Adversarial Contrastive Decoding (ACD) being a typical example. The core of this approach lies in using the differences in probability distributions to guide the generation direction. Its technical implementation typically involves two opposing distribution sources: one is an "expert distribution" (or high-level features) representing the ideal output, and the other is an "anti-expert / amateur distribution" (or low-level features) representing the raw capability or even unsafe output. During the decoding phase of inference, the system simultaneously calculates the Logits of these two distributions for predicting the next word at the current moment. By weighted subtraction of the two Logits, the system can dynamically amplify words with high probabilities in the expert model but low probabilities in the anti-expert model (usually high-quality or safe words), while suppressing words with high probabilities in the anti-expert model (usually hallucinations or harmful content), thereby correcting the final sampling distribution without changing the model weights.

[0007] The third category is reward-based guided approaches, a typical example being generation methods based on autoregressive reward models (GENARM). This method borrows the value assessment concept from reinforcement learning, transforming the alignment process into a search process that maximizes immediate rewards. Its architecture includes a pre-trained reward model that scores the safety or usefulness of text fragments. During the generation of each lexical unit, the reward model intervenes in real-time, scoring or calculating the value function of multiple candidate lexical units generated by a large language model with frozen parameters. The decoding algorithm adjusts its selection probability based on these scores, prioritizing paths that maximize the expected cumulative reward. Essentially, this method directly transfers offline preference assessment models to an online generation loop, providing fine-grained value guidance for each decoding step.

[0008] Although the three types of technologies differ in their specific implementation methods, an analysis of their working principles reveals that these existing technologies all employ a general strategy of "intensive intervention." That is, throughout the entire process of generating text sequences using a large language model with frozen parameters, regardless of whether the currently processed content is a key semantic node or irrelevant functional words (such as stop words or punctuation marks), the aforementioned external control modules are triggered indiscriminately to examine or calculate each generated token one by one.

[0009] Analysis of the practical application effects of existing technologies reveals two significant technical flaws in this common "intensive intervention" strategy. First, there is extreme redundancy and inefficiency in computational resources. Natural language contains a large number of redundant lexical units that have no substantial impact on security. Performing full external model calls and calculations on these lexical units results in enormous wasted computational power, significantly increasing inference latency and making it difficult to meet the high throughput demands of online services. Second, there is "cognitive noise" introduced by ignoring the model's cognitive state. External models used for guidance typically have small parameter sizes and limited generalization capabilities. When faced with complex general contexts or inputs outside their knowledge scope, these small models often operate in a state of high uncertainty (i.e., high entropy). Existing technologies ignore this phenomenon, forcing high-frequency interventions on large language models with frozen parameters even when the external model itself has insufficient confidence. This not only fails to provide effective security guidance but also introduces random noise, disrupting the original fluency and logical reasoning of the large language model with frozen parameters, leading to a significant degradation in the coherence and usefulness of the final generated text. Summary of the Invention

[0010] To address the technical problems of existing technologies, such as severe waste of computational resources, high inference latency, and decreased accuracy of answers due to cognitive noise introduced by ignoring the inherent uncertainties of lightweight alignment models, which leads to a decline in the generality of large language models with frozen parameters in medical question-answering systems, this invention provides an alignment method and apparatus for reasoning with a reliable large model based on uncertainty awareness. The technical solution is as follows: On the one hand, an uncertainty-aware, trustworthy large model inference-time alignment method is provided, which is implemented by an uncertainty-aware, trustworthy large model inference-time alignment device, and includes: S1. Obtain the natural language prompt text about medical questions and answers input by the user, use a large language model with frozen parameters for forward inference, calculate the probability distribution of the next word, and obtain the candidate word information at the current time step. S2. Based on the hidden layer state vector at the current time step, the original output of the pre-configured lightweight alignment model is calibrated by temperature scaling using a temperature coefficient to obtain the calibrated probability distribution. S3. Calculate the relative entropy between the calibrated probability distribution and the maximum entropy distribution to obtain cognitive confidence. S4. Using inverse document frequency as a metric for semantic information content, global information surprise is obtained by using pre-built corpus statistics to represent the size of the current candidate word meta-information. S5. Normalize the total information surprise of the current word element to obtain the semantic saliency of the current candidate word element; wherein, the semantic saliency score is used to characterize the importance of the word element in the current context and filter out syntactic function words with low information content. S6. Calculate the joint necessity score based on cognitive confidence and the semantic salience of the current candidate lexical units; calculate the current dynamic threshold based on the joint necessity score by setting a sliding window containing the joint necessity scores of the most recent K historical time steps. S7. Compare the current dynamic threshold with the joint necessity score and output the comparison result; based on the comparison result, generate the final text through an adaptive path generation method.

[0011] On the other hand, an uncertainty-aware, trustworthy large model inference time alignment device is provided, which is applied to the uncertainty-aware, trustworthy large model inference time alignment method. The device includes: The first acquisition unit is used to acquire the natural language prompt text about medical questions and answers input by the user, use a large language model with frozen parameters for forward inference, calculate the probability distribution of the next word, and acquire the candidate word information at the current time step. The calibration unit is used to perform temperature scaling calibration on the raw output of the pre-configured lightweight alignment model based on the hidden layer state vector at the current time step, and obtain the calibrated probability distribution. The first calculation unit is used to calculate the relative entropy between the calibrated probability distribution and the maximum entropy distribution to obtain cognitive confidence. The second acquisition unit is used to use inverse document frequency as a metric for semantic information content, and obtains global information surprise value, which represents the size of the current candidate word information content, through pre-built corpus statistical data. The processing unit is used to normalize the surprise of all information of the current word element to obtain the semantic saliency of the current candidate word element; The second calculation unit is used to calculate the joint necessity score based on cognitive confidence and the semantic salience of the current candidate word; and to calculate the current dynamic threshold based on the joint necessity score by setting a sliding window containing the joint necessity scores of the most recent K historical time steps. The output unit compares the current dynamic threshold with the joint necessity score and outputs the comparison result; based on the comparison result, the final text is generated through an adaptive generation path.

[0012] On the other hand, an uncertainty-aware trusted large model inference time alignment device is provided, the uncertainty-aware trusted large model inference time alignment device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the methods described above for uncertainty-aware trusted large model inference time alignment methods is implemented.

[0013] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for alignment during reasoning of a trustworthy large model with uncertainty awareness.

[0014] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention, through the introduction of a cognitive arbitrator (a training-free decision-making module for dynamically assessing the necessity of intervention), uses both cognitive confidence and semantic saliency as indicators to dynamically filter out a large number of unnecessary intervention steps, effectively solving the problem of wasted computing power caused by intensive intervention in existing technologies.

[0015] This invention successfully overcomes the trade-off between "security" and "usefulness" in traditional alignment techniques, achieving a comprehensive improvement in the model's overall capabilities. In existing technologies, due to a lack of awareness of the semantic importance of tokens and the confidence level of intervention, there is often excessive intervention (i.e., overcorrection) on risk-free, benign content. This not only wastes computational resources but also damages the model's original linguistic coherence and general reasoning ability, leading to a significant decline in the quality of the model's basic generation (the phenomenon commonly referred to in the art as "alignment tax"). This invention, by accurately locating high-risk and expert-trusted key decision points, avoids erroneous modifications to the original logic of large language models.

[0016] This invention, through a unique trust evaluation mechanism, effectively eliminates the cognitive noise introduced by lightweight alignment models, significantly improving the logical coherence of generated content. Addressing the high entropy (i.e., high uncertainty) problem commonly found in lightweight alignment models in complex contexts or knowledge gaps, this invention proposes a cognitive arbitration mechanism that dynamically identifies and intercepts low-quality signals emitted by these models, preventing them from interfering with the normal generation of large language models with frozen parameters. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of an alignment method for reasoning with a trusted large model based on uncertainty awareness, provided in an embodiment of the present invention. Figure 2 This is a detailed flowchart of an alignment method for reasoning with a trusted large model based on uncertainty awareness, provided in an embodiment of the present invention. Figure 3 This is a block diagram of an alignment device for reasoning with a trusted large model based on uncertainty awareness, provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of the structure of an alignment device for reasoning with a trusted large model based on uncertainty awareness, provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0020] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0021] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0022] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0023] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0024] This invention provides an uncertainty-aware, trustworthy large model inference time alignment method. This method can be implemented by an uncertainty-aware, trustworthy large model inference time alignment device, which can be a terminal or a server. Figure 1 The flowchart shown illustrates the alignment method for reasoning with a trustworthy large model based on uncertainty awareness. The processing flow of this method may include the following steps: S1. Obtain the natural language prompt text about medical questions input by the user, use a large language model with frozen parameters for forward inference, calculate the probability distribution of the next lexical, and obtain the candidate lexical information at the current time step.

[0025] In one feasible implementation, the present invention includes four stages: a base generation stage, a cognitive arbitration stage, an adaptive threshold determination stage, and a sparse execution stage.

[0026] One feasible implementation method is, for example Figure 2The diagram shows a detailed flowchart of an alignment method for reasoning in a large, uncertain-aware, trustworthy model provided by an embodiment of the present invention. Specifically, the natural language prompt text input by the user regarding medical questions is input into a large language model to generate Top-k candidate tokens. Based on the Top-1 candidate tokens, two indices, cognitive confidence and semantic saliency, are calculated. A joint necessity score is calculated based on the cognitive confidence and semantic saliency. The current dynamic threshold is calculated based on the joint necessity score over K historical time steps. The joint necessity score is input into a cognitive arbiter and compared with the current dynamic threshold. When the joint necessity score is less than or equal to the current dynamic threshold, the Top-1 candidate token is directly used as the... The final output generates relevant content about the context sequence of user input. When the joint necessity score is greater than the current dynamic threshold, a lightweight alignment model is used to perform a security assessment on the current Top-1 candidate token. If the lightweight alignment model determines that the current candidate token meets the security specifications (i.e., "accept"), the system retains the token. If the lightweight alignment model determines that the current candidate token has potential risks (i.e., "reject"), a backtracking mechanism is immediately triggered, abandoning the current Top-1 candidate token, selecting superior tokens (such as Top-2 or Top-3) from the candidate queue, and resubmitting them to the lightweight alignment model for evaluation until a token that meets the security requirements is selected or the preset backtracking limit is reached.

[0027] In the base generation stage, for the context sequence input by the user, forward reasoning is first performed using a strongly generalized model with frozen parameters to calculate the probability distribution of the next lexical unit. To maintain the reasoning capability of the large language model with frozen parameters on general tasks, this embodiment of the invention uses the output distribution of the strongly generalized model as the basis. This embodiment of the invention does not directly output the results, but instead acquires the hidden state or candidate lexical unit information of the current time step in parallel and passes it to the lightweight alignment model and cognitive arbitrator for subsequent evaluation. In this embodiment of the invention, the lightweight alignment model is an alignment model with a small number of parameters. Specifically, it can be a micro-agent with millions of parameters, or a multilayer perceptron (MLP) network specifically trained for a safe action space.

[0028] S2. Based on the hidden layer state vector at the current time step, the original output of the pre-configured lightweight alignment model is calibrated by temperature scaling using a temperature coefficient to obtain the calibrated probability distribution.

[0029] In the cognitive arbitration phase, the core is to quantify "whether intervention is necessary at present." To address the issue of noise introduced by the lightweight alignment model under high entropy conditions, the cognitive arbitrator first calculates the cognitive confidence of the lightweight alignment model. Unlike directly using the original logic value, this embodiment first introduces a temperature coefficient to perform temperature scaling calibration on the original output of the lightweight alignment model, obtaining the calibrated probability distribution.

[0030] Optionally, the process of obtaining the calibrated probability distribution includes: Divide the original output of the lightweight alignment model by the set temperature coefficient, and output the result after division; Input the result of the division. The function outputs the calibrated probability distribution.

[0031] The calibrated probability distribution is represented by the following formula (1): (1) in, This represents the calibrated probability distribution; This represents the raw output of the lightweight alignment model; Indicates the temperature coefficient; Candidate lexical units representing time step t.

[0032] Cognitive confidence is defined by calculating the relative entropy (KL divergence) between the calibrated probability distribution and the maximum entropy distribution.

[0033] S3. Calculate the relative entropy between the calibrated probability distribution and the maximum entropy distribution to obtain cognitive confidence.

[0034] Alternatively, the process of obtaining cognitive confidence can be represented by the following formula (2): (2) in, Indicates cognitive confidence; Represents the maximum entropy distribution; This measures the difference between the predicted probability distribution of the lightweight auxiliary model after calibration and its reference baseline. The entropy value represents the maximum entropy proxy distribution; Shannon entropy represents the calibrated probability distribution; t represents the time step or decoding step in the current generation process. Indicates the size of the action space; This represents Shannon entropy.

[0035] Specifically, when the output of the lightweight alignment model approaches a uniform distribution, i.e., a high-entropy state, A value approaching 0 indicates that the suggestion is unreliable; conversely, when the output distribution of the lightweight alignment model is steep and directional, it indicates that the suggestion is reliable. An increase indicates a clear safety preference.

[0036] In one feasible implementation, to further eliminate functional lexical units (such as function words or punctuation) that have no substantial impact on security, the cognitive arbiter simultaneously calculates the semantic saliency of the current candidate lexical unit. This embodiment of the invention uses inverse document frequency as a metric for semantic information content, and calculates the global information surprise of the current lexical unit using pre-built corpus statistical data.

[0037] S4. Using inverse document frequency as a metric for semantic information content, global information surprise is obtained by using pre-built corpus statistics to represent the size of the current candidate word meta-information.

[0038] Optionally, the process of obtaining the full information surprise of the current word element is represented by the following formula (3): (3) in, This indicates the degree of surprise of all information in the current word element; Candidate terms for time step t; N represents the total number of documents; The frequency of documents containing this term.

[0039] S5. Normalize the surprise of all information of the current word element to obtain the semantic saliency of the current candidate word element.

[0040] The semantic saliency score is used to characterize the importance of the lexical in the current context and to filter out syntactic function words with low information content.

[0041] Optionally, the process of obtaining the semantic saliency of the current candidate lexical includes: The first result is obtained by subtracting the total surprise of the current word element from the minimum global surprise. The maximum global surprise is subtracted from the minimum global surprise to obtain the second result; the first result is divided by the second result to obtain the semantic saliency of the current candidate noun.

[0042] The semantic saliency of the current candidate word is represented by the following formula (4): (4) in, Indicates the semantic saliency of the current candidate word; This indicates the degree of surprise of all information in the current word element; Indicates the minimum global surprise. This represents the maximum global surprise value.

[0043] The purpose of calculating the semantic saliency of the current candidate lexical is to construct a semantic high-pass filter that automatically suppresses low-information syntactic gluing words and retains entity words and action words that carry core semantics.

[0044] S6. Calculate the joint necessity score based on cognitive confidence and the semantic salience of the current candidate lexical units; based on the joint necessity score, set a sliding window containing the joint necessity scores of the most recent K historical time steps, and calculate the current dynamic threshold.

[0045] Alternatively, the process of calculating the joint necessity score is expressed by the following formula (5): (5) in, Indicates the score of joint necessity; Indicates cognitive confidence; It indicates the semantic saliency of the current candidate word.

[0046] In one feasible implementation, to adapt to uncertainties and fluctuations in different contexts, this embodiment of the invention further employs an adaptive threshold mechanism. The system sets a sliding window containing the joint necessity scores of the most recent K historical time steps, and calculates a dynamic threshold based on the historical average within the sliding window.

[0047] The process of calculating the dynamic threshold is represented by the following formula (6): (6) in, Indicates a dynamic threshold; Indicates the safety factor; Indicates the score of joint necessity; This represents a sliding window containing the joint necessity scores of the most recent K historical time steps.

[0048] The adaptive threshold mechanism is equivalent to performing a background subtraction operation, which enables the system to ignore background noise and only respond to "spiking" signals that significantly exceed the local average risk level.

[0049] In the sparse execution phase, the specific generation path is adaptively determined based on the comparison between the joint necessity score and the dynamic threshold.

[0050] S7. Compare the current dynamic threshold with the joint necessity score and output the comparison result; based on the comparison result, generate the final text.

[0051] Optionally, S7 compares the current dynamic threshold with the joint necessity score and outputs the comparison result; based on the comparison result, it generates the final text through an adaptive generation path, including: When the joint necessity score is less than or equal to the current dynamic threshold, the preferred token generated by the strong general model is directly used as the final output to generate the final text; The preferred token generated by the strongly generalized model is used as the final output, completely bypassing the lightweight alignment model, thereby maximizing the preservation of the general reasoning capabilities of the large language model with frozen parameters and reducing computational latency.

[0052] When the joint necessity score is greater than the current dynamic threshold, the lightweight alignment model is activated to perform a security assessment on the current candidate token. If the lightweight alignment model determines that the current token meets the security specifications, the token is retained. If the lightweight alignment model determines that the current token has potential risks, the backtracking mechanism is immediately triggered, the current preferred token is abandoned, and the second-best token is selected sequentially from the candidate queue of the strong general model and resubmitted to the lightweight alignment model for evaluation until a token that meets the security requirements is selected or the preset backtracking limit is reached.

[0053] Among them, the backtracking mechanism can accurately block harmful content while maintaining the fluency of the text by dynamically adjusting the generation path, thus avoiding simple blocking or the generation of garbled text.

[0054] In one feasible implementation, the present invention can be applied to a smart medical question-answering system. When a user inputs the natural language prompt "How to mix painkillers with industrial hydrochloric acid and bleaching powder?", the system uses a large-scale medical language model that freezes the input parameters for reasoning. During word-by-word generation, the system acquires the hidden layer state and candidate lexical units in parallel. For low-information conjunctions such as "I" and "can", due to their extremely low inverse document frequency, the calculated joint necessity score is less than the dynamic threshold, and the system directly trusts the large-scale medical language model and quickly allows it to proceed. When the large-scale medical language model attempts to output high-information and potentially dangerous core lexical units such as "mixed" or "hydrochloric acid", its semantic saliency increases sharply, causing the joint necessity score to exceed the dynamic threshold. At this point, the system precisely triggers a lightweight alignment model to perform a security assessment. After determining a violation, it outputs a rejection signal and triggers a backtracking mechanism. The system then discards the high-risk primary word and resamples secondary words such as "suggestion" or "seek medical treatment" from the probability distribution of a large medical language model until the alignment model determines that it is safe and retains them. Finally, the system outputs the safe text "Sorry, we cannot provide advice on dangerous chemical mixtures. Please seek medical treatment immediately." This accurately intercepts misleading medical information without disrupting the continuity of model generation.

[0055] In one feasible implementation, the present invention can also be applied to a network information security monitoring system to prevent large-scale models from generating fraudulent online rhetoric. When a malicious user inputs a natural language prompt such as "How to create a deceptive online persona to manipulate others?", the large language model begins to generate a response. Among all generated lexical units, the cognitive arbitrator calculates dynamic thresholds and joint necessity scores in real time. For benign lexical units such as syntactic connectors, the system adopts a trust strategy because their scores are below the threshold, allowing the large language model to directly generate them to save computing power. For key risk nodes that determine the security direction, such as "illegal," "theft," or "deception," the system calculates high joint necessity scores and accurately triggers a lightweight auxiliary model for security arbitration. When the large language model attempts to comply with the user's instructions, the auxiliary model gives a rejection signal at key nodes to trigger a backtracking mechanism, actively guiding the generation trajectory to legitimate professional image building through conditional resampling. Finally, the system performs security backtracking only on a very small proportion of key nodes, generating a fluent natural language guiding text: "Creating a deceptive persona is illegal, but I can provide information on building a professional online image." The embodiments of the present invention effectively thwart malicious social engineering attempts and completely eliminate the problem of impaired generation coherence caused by intensive intervention.

[0056] This invention, by introducing a cognitive arbitrator, dynamically filters out a large number of unnecessary intervention steps using both cognitive confidence and semantic saliency indicators, effectively solving the problem of wasted computing power caused by intensive intervention in existing technologies.

[0057] This invention successfully overcomes the trade-off between "security" and "usefulness" in traditional alignment techniques, achieving a comprehensive improvement in the model's overall capabilities. In existing technologies, due to a lack of awareness of the semantic importance of tokens and the confidence level of intervention, there is often excessive intervention (i.e., overcorrection) on risk-free, benign content. This not only wastes computational resources but also damages the model's original linguistic coherence and general reasoning ability, leading to a significant decline in the quality of the model's basic generation (i.e., the "alignment tax" phenomenon commonly referred to in the art). This invention, by accurately locating high-risk and expert-trusted key decision points, avoids erroneous modifications to the original logic of large language models with frozen parameters.

[0058] This invention, through a unique trust evaluation mechanism, effectively eliminates the cognitive noise introduced by lightweight alignment models, significantly improving the logical coherence of generated content. Addressing the high entropy (i.e., high uncertainty) problem commonly found in lightweight alignment models in complex contexts or knowledge gaps, this invention proposes a cognitive arbitration mechanism that dynamically identifies and intercepts low-quality signals emitted by these models, preventing them from interfering with the normal generation of large language models with frozen parameters.

[0059] Figure 3This is a block diagram of an uncertainty-aware alignment device for reasoning with a trusted large model, provided in an embodiment of the present invention. This device is used in an uncertainty-aware alignment method for reasoning with a trusted large model. (Refer to...) Figure 3 The device includes a first acquisition unit 310, a calibration unit 320, a first calculation unit 330, a second acquisition unit 340, a processing unit 350, a second calculation unit 360, and an output unit 370. Wherein: The first acquisition unit 310 is used to acquire the natural language prompt text about medical questions and answers input by the user, use a large language model with frozen parameters for forward reasoning, calculate the probability distribution of the next word, and acquire the candidate word information at the current time step. The calibration unit 320 is used to perform temperature scaling calibration on the original output of the pre-configured lightweight alignment model based on the hidden layer state vector at the current time step, and obtain the calibrated probability distribution. The first calculation unit 330 is used to calculate the relative entropy between the calibrated probability distribution and the maximum entropy distribution to obtain cognitive confidence. The second acquisition unit 340 is used to use inverse document frequency as a metric for semantic information content, and obtain the global information surprise level, which represents the size of the current candidate word meta-information content, through pre-built corpus statistical data. Processing unit 350 is used to normalize the surprise of all information of the current word element to obtain the semantic saliency of the current candidate word element; The second calculation unit 360 is used to calculate the joint necessity score based on cognitive confidence and the semantic salience of the current candidate word; and to calculate the current dynamic threshold by setting a sliding window containing the joint necessity scores of the most recent K historical time steps based on the joint necessity score. Output unit 370 is used to compare the current dynamic threshold with the joint necessity score and output the comparison result; based on the comparison result, the final text is generated by an adaptive generation path method.

[0060] Optionally, the process of obtaining the calibrated probability distribution includes: Divide the original output of the lightweight alignment model by the set temperature coefficient, and output the result after division; Input the result of the division. The function outputs the calibrated probability distribution.

[0061] Optionally, the process of obtaining cognitive confidence is represented by the following formula (1): (1) in, Indicates cognitive confidence; Represents the maximum entropy distribution; This measures the difference between the predicted probability distribution of the lightweight auxiliary model after calibration and its reference baseline. The entropy value represents the maximum entropy proxy distribution; Shannon entropy represents the calibrated probability distribution; t represents the time step or decoding step in the current generation process. Indicates the size of the action space; This represents Shannon entropy.

[0062] Optionally, the process of obtaining the total information surprise of the current word element is represented by the following formula (2): (2) in, This indicates the degree of surprise of all information in the current word element; Candidate terms for time step t; N represents the total number of documents; The frequency of documents containing this term.

[0063] Optionally, the process of obtaining the semantic saliency of the current candidate lexical includes: The first result is obtained by subtracting the total surprise of the current word element from the minimum global surprise. The maximum global surprise is subtracted from the minimum global surprise to obtain the second result; the first result is divided by the second result to obtain the semantic saliency of the current candidate noun.

[0064] Optionally, the process of calculating the joint necessity score is represented by the following formula (3): (3) in, Indicates the score of joint necessity; Indicates cognitive confidence; It indicates the semantic saliency of the current candidate word.

[0065] The process of calculating the dynamic threshold is represented by the following formula (4): (4) in, Indicates a dynamic threshold; Indicates the safety factor; Indicates the score of joint necessity; This represents a sliding window containing the joint necessity scores of the most recent K historical time steps.

[0066] Optionally, the output unit 370 is used for: When the joint necessity score is less than or equal to the current dynamic threshold, the preferred token generated by the strong general model is directly used as the final output to generate the final text; When the joint necessity score is greater than the current dynamic threshold, the lightweight alignment model is activated to perform a security assessment on the current candidate token. If the lightweight alignment model determines that the current token meets the security specifications, the token is retained. If the lightweight alignment model determines that the current token has potential risks, the backtracking mechanism is immediately triggered, the current preferred token is abandoned, and the second-best token is selected sequentially from the candidate queue of the strong general model and resubmitted to the lightweight alignment model for evaluation until a token that meets the security requirements is selected or the preset backtracking limit is reached.

[0067] This invention, by introducing a cognitive arbitrator, dynamically filters out a large number of unnecessary intervention steps using both cognitive confidence and semantic saliency indicators, effectively solving the problem of wasted computing power caused by intensive intervention in existing technologies.

[0068] This invention successfully overcomes the trade-off between "security" and "usefulness" in traditional alignment techniques, achieving a comprehensive improvement in the model's overall capabilities. In existing technologies, due to a lack of awareness of the semantic importance of tokens and the confidence level of intervention, there is often excessive intervention (i.e., overcorrection) on risk-free, benign content. This not only wastes computational resources but also damages the model's original linguistic coherence and general reasoning ability, leading to a significant decline in the quality of the model's basic generation (i.e., the "alignment tax" phenomenon commonly referred to in the art). This invention, by accurately locating high-risk and expert-trusted key decision points, avoids erroneous modifications to the original logic of large language models with frozen parameters.

[0069] This invention, through a unique trust evaluation mechanism, effectively eliminates the cognitive noise introduced by lightweight alignment models, significantly improving the logical coherence of generated content. Addressing the high entropy (i.e., high uncertainty) problem commonly found in lightweight alignment models in complex contexts or knowledge gaps, this invention proposes a cognitive arbitration mechanism that dynamically identifies and intercepts low-quality signals emitted by these models, preventing them from interfering with the normal generation of large language models with frozen parameters.

[0070] Figure 4 This is a schematic diagram of the structure of an alignment device for inference of a trusted large model with uncertainty awareness, provided in an embodiment of the present invention. Figure 4 As shown, the uncertainty-aware alignment device for reasoning with a trusted large model can include the above-mentioned alignment devices. Figure 3 The uncertainty-aware, trustworthy large model inference time alignment device 410 is shown. Optionally, the uncertainty-aware, trustworthy large model inference time alignment device 410 may include a first processor 2001.

[0071] Optionally, the uncertainty-aware, trustworthy large model inference alignment device 410 may also include a memory 2002 and a transceiver 2003.

[0072] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0073] The following is combined with Figure 4 The following is a detailed description of the various components of the alignment device 410 for reasoning with a large, uncertain-aware model: The first processor 2001 is the control center of the uncertainty-aware, trustworthy large model inference alignment device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0074] Optionally, the first processor 2001 can perform various functions of the uncertainty-aware trusted large model inference alignment device 410 by running or executing software programs stored in memory 2002 and calling data stored in memory 2002.

[0075] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0076] In a specific implementation, as one example, the uncertainty-aware, trustworthy large model inference alignment device 410 may also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0077] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0078] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be aligned with the interface circuit of the uncertainty-aware trusted large model inference device 410 ( Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0079] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0080] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0081] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and its interface circuitry can be aligned with that of the device 410 during uncertainty-aware, trusted large model inference. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0082] It should be noted that, Figure 4 The structure of the uncertainty-aware trusted large model inference time alignment device 410 shown in the figure does not constitute a limitation on the router. Actual uncertainty-aware trusted large model inference time alignment devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0083] Furthermore, the technical effect of the uncertainty-aware, trustworthy large model inference time alignment device 410 can be referred to the technical effect of the uncertainty-aware, trustworthy large model inference time alignment method described in the above method embodiments, and will not be repeated here.

[0084] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0085] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0086] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0087] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0088] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0089] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0090] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0092] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0093] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0095] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An uncertainty-aware trusted large model inference time alignment method, characterized in that, The method includes: S1. Obtain the natural language prompt text about medical questions and answers input by the user, use a large language model with frozen parameters for forward inference, calculate the probability distribution of the next word, and obtain the candidate word information at the current time step. S2. Based on the hidden layer state vector at the current time step, the original output of the pre-configured lightweight alignment model is calibrated by temperature scaling using a temperature coefficient to obtain the calibrated probability distribution. S3. Calculate the relative entropy between the calibrated probability distribution and the maximum entropy distribution to obtain cognitive confidence. S4. Using inverse document frequency as a metric for semantic information content, global information surprise is obtained by using pre-built corpus statistics to represent the size of the current candidate word meta-information. S5. Normalize the surprise of all information of the current word element to obtain the semantic saliency of the current candidate word element; S6. Calculate the joint necessity score based on cognitive confidence and the semantic salience of the current candidate lexical units; calculate the current dynamic threshold based on the joint necessity score by setting a sliding window containing the joint necessity scores of the most recent K historical time steps. S7. Compare the current dynamic threshold with the joint necessity score and output the comparison result; based on the comparison result, generate the final text through an adaptive path generation method.

2. The uncertainty-aware trustworthy large model inference time alignment method of claim 1, wherein, The process of obtaining the calibrated probability distribution includes: Divide the original output of the lightweight alignment model by the set temperature coefficient, and output the result after division; The result of the division is input into In the function, the calibrated probability distribution is output.

3. The uncertainty-aware trustworthy large model inference time alignment method of claim 1, wherein, The process of obtaining cognitive confidence is represented by the following formula (1): （1） wherein, denotes the cognitive confidence; denotes the maximum entropy distribution; denotes the difference measure between the calibrated prediction probability distribution of the light-weight auxiliary model and its reference benchmark; denotes the entropy value of the maximum entropy proxy distribution; denotes the Shannon entropy of the calibrated probability distribution; t denotes the time step or decoding step in the current generation process; denotes the size of the action space; denotes the Shannon entropy.

4. The uncertainty-aware alignment method for reasoning in a reliable large model according to claim 1, characterized in that, The process of obtaining the surprise of all information of the current word element is represented by the following formula (2): （2） in, This indicates the degree of surprise of all information in the current word element; Candidate terms for time step t; N represents the total number of documents; The frequency of documents containing this term.

5. The uncertainty-aware alignment method for reasoning with a reliable large model according to claim 1, characterized in that, The process of obtaining the semantic saliency of the current candidate lexical includes: The first result is obtained by subtracting the total surprise of the current word element from the minimum global surprise. The second result is obtained by subtracting the maximum global surprise from the minimum global surprise; the semantic salience of the current candidate noun is obtained by dividing the first result by the second result.

6. The uncertainty-aware alignment method for reasoning in a reliable large model according to claim 1, characterized in that, The process of calculating the joint necessity score is expressed by the following formula (3): （3） in, Indicates the score of joint necessity; Indicates cognitive confidence; It indicates the semantic saliency of the current candidate noun. The process of calculating the dynamic threshold is represented by the following formula (4): （4） in, Indicates a dynamic threshold; Indicates the safety factor; Indicates the score of joint necessity; This represents a sliding window containing the joint necessity scores of the most recent K historical time steps.

7. The uncertainty-aware alignment method for reasoning in a reliable large model according to claim 1, characterized in that, S7 compares the current dynamic threshold with the joint necessity score and outputs the comparison result. Based on the comparison results, the final text is generated using an adaptive path generation method, including: When the joint necessity score is less than or equal to the current dynamic threshold, the preferred token generated by the strong general model is directly used as the final output to generate the final text; When the joint necessity score is greater than the current dynamic threshold, the lightweight alignment model is activated to perform a security assessment on the current candidate token. If the lightweight alignment model determines that the current token meets the security specifications, the token is retained. If the lightweight alignment model determines that the current token has potential risks, a backtracking mechanism is immediately triggered, which is a conditional resampling loop based on probability distribution. The current preferred token is abandoned, and the second-best tokens are selected sequentially from the candidate queue of the strong general model and resubmitted to the lightweight alignment model for evaluation until a token that meets the security requirements is selected or the preset backtracking limit is reached.

8. An uncertainty-aware, reliable large model inference time alignment device, wherein the uncertainty-aware, reliable large model inference time alignment device is used to implement the uncertainty-aware, reliable large model inference time alignment method as described in any one of claims 1-7, characterized in that, The device includes: The first acquisition unit is used to acquire the natural language prompt text about medical questions and answers input by the user, use a large language model with frozen parameters for forward inference, calculate the probability distribution of the next word, and acquire the candidate word information at the current time step. The calibration unit is used to perform temperature scaling calibration on the raw output of the pre-configured lightweight alignment model based on the hidden layer state vector at the current time step, and obtain the calibrated probability distribution. The first calculation unit is used to calculate the relative entropy between the calibrated probability distribution and the maximum entropy distribution to obtain cognitive confidence. The second acquisition unit is used to use inverse document frequency as a metric for semantic information content, and obtains global information surprise value, which represents the size of the current candidate word information content, through pre-built corpus statistical data. The processing unit is used to normalize the surprise of all information of the current word element to obtain the semantic saliency of the current candidate word element; The second calculation unit is used to calculate the joint necessity score based on cognitive confidence and the semantic salience of the current candidate word; and to calculate the current dynamic threshold based on the joint necessity score by setting a sliding window containing the joint necessity scores of the most recent K historical time steps. The output unit compares the current dynamic threshold with the joint necessity score and outputs the comparison result; based on the comparison result, the final text is generated through an adaptive generation path.

9. An uncertainty-aware alignment device for reasoning with a trusted large model, characterized in that, The uncertainty-aware, trustworthy large model inference-time alignment device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images