A large language model output processing method and system based on local soft margin

Abstract

The application provides a large language model output processing method and system based on local soft margin, relating to the technical field of artificial intelligence, comprising: generating a target sequence through single forward inference, and extracting an original Logit score vector of each position without Softmax in the generation process; after local candidate interception, obtaining a local soft margin score based on the flexible aggregation of the target candidate and the suboptimal candidate set, and then obtaining a sequence-level uncertainty quantification score through sequence-level aggregation; finally, comparing with a preset safety threshold, triggering downstream business if the score is high, and blocking the output and routing to a safety fallback channel if the score is low. The method is based on the original Logit space throughout and does not require multiple sampling, and both real-time performance and reliability are taken into account.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a method and system for processing the output of a large language model based on local soft margins. Background Technology

[0002] With the continuous improvement of large language models' capabilities, they have been widely applied in scenarios such as intelligent question answering, user intent recognition, text classification, knowledge extraction, and automated business processing. However, large language models commonly suffer from the "illusion" problem in practical applications, meaning that even with insufficient knowledge, ambiguous input, or unclear semantic boundaries, the model may still output seemingly fluent but factually or logically flawed content. Since such errors are usually difficult to identify directly from the surface form of the output, a single reasoning error can lead to business losses, incorrect decisions, or security risks in scenarios with high concurrency, high automation, and high security requirements. Therefore, how to assess the reliability of large language model outputs and promptly trigger rollback, rejection, or escalation when output credibility is insufficient has become a key technical issue in the trusted deployment of large language models.

[0003] To address the aforementioned issues, existing technologies have proposed schemes for quantifying the uncertainty of large language model outputs. This involves assigning a quantified score reflecting the reliability of each model output, enabling upper-level systems to perform subsequent processing such as approval, rollback, model cascading, or manual intervention based on this score. Current related technologies generally fall into two categories: one is the black-box approach, which typically involves generating multiple independent outputs from the same input and then indirectly inferring model uncertainty based on the consistency or semantic differences between the multiple outputs; the other is the white-box approach, which directly utilizes output layer information obtainable during model inference, such as token probabilities, Logit scores, or relevant statistics, to assess the uncertainty of a single inference result.

[0004] While black-box methods can reflect model instability from the differences between multiple sampling results, they typically require multiple autoregressive generation operations on the same input. Furthermore, in many scenarios, auxiliary models are needed to perform semantic equivalence judgments, clustering, or consistency analysis on multiple outputs, leading to a significant increase in inference latency and computational overhead with the number of samplings. These methods usually achieve good detection results in offline evaluations, but their computational cost and response latency are often unacceptable in high-concurrency, low-latency real-world deployment scenarios.

[0005] In contrast, white-box methods can directly obtain uncertainty quantification results under single-inference conditions, which is more conducive to engineering implementation and thus has become a more attractive technical direction in current practical systems. Among existing white-box methods, one type only utilizes the probability or score of the top-1 candidate tokens, such as the minimum token probability or sequence probability; another type utilizes the difference between the first and second ranked candidate tokens, i.e., the so-called probability margin or a similar form of margin score. In addition, existing technologies have proposed uncertainty quantification methods based on full vocabulary entropy, perplexity, or Top-k local distribution statistics, aiming to improve evaluation capabilities by introducing more candidate distribution information.

[0006] However, existing technologies still have significant shortcomings. Methods that only utilize the probability or score of the Top-1 candidate fail to reflect the competitive relationship between candidates at the current output position because they only observe the local advantage of a single candidate. Therefore, they may still give an overly high certainty judgment when the model exhibits significant hesitation. Methods based on the difference between Top-1 and Top-2 candidates, while reflecting the difference between the top two candidates, still struggle to fully characterize the uncertainty at the current position when multiple suboptimal candidates simultaneously have high scores.

[0007] On the other hand, while methods based on full vocabulary entropy, perplexity, or Top-k local distribution statistics expand the candidate observation range, they also introduce new problems. Full vocabulary probability distributions typically contain a large number of low-probability candidates irrelevant to the actual prediction at the current position. These long-tailed candidates increase computational burden and interfere with statistical results. To reduce long-tail noise, existing techniques often employ Top-k truncation, retaining only the top-ranked candidate tokens before performing distribution statistics on the truncated candidate set. However, in this process, existing techniques usually require re-performing Softmax normalization on the truncated candidate scores to form a local probability distribution. While this process ensures the truncated candidate set satisfies probability distribution constraints, it also compresses the absolute score differences in the original Logit space into local relative proportions, thus affecting the ability of the uncertainty quantification results to reflect the true confidence level.

[0008] Furthermore, in sequence output scenarios, the uncertainty of a single generation position cannot be directly equated to the uncertainty of the entire output sequence, thus requiring further sequence-level aggregation. However, simple aggregation methods such as summation and averaging are easily affected by sequence length and may mask the constraining effect of individual high-risk generation positions on the overall output credibility. Meanwhile, employing more complex multiple sampling or post-processing strategies introduces problems such as high computational cost and high system implementation complexity. Therefore, how to accurately quantify the uncertainty of large language model output under single-inference conditions and effectively use it for output-level credibility decision-making remains a problem to be solved by existing technologies. Summary of the Invention

[0009] This application provides a reliable processing method for the output of a large language model based on local soft margins to solve the above-mentioned technical problems.

[0010] The technical solution adopted in this application is as follows: This application provides a method for reliable output processing of large language models based on local soft margins, including: Receive natural language input and control a large language model to perform a single forward inference based on the natural language input to generate a target output sequence; During the generation of the target output sequence, the original Logit score vector, which is not Softmax normalized, is extracted from the output of the large language model at each generation position. Local candidate extraction processing is performed on the original Logit score vector corresponding to each generation position to select a preset number of candidate tokens with the highest scores from all candidates corresponding to the vocabulary, forming a local candidate set for the corresponding generation position, and determining the target candidate ranked first and the set of second-best candidates other than the target candidate in the local candidate set. Based on the original Logit score of the target candidate and the original Logit score of each candidate token in the suboptimal candidate set, the comprehensive competitive relationship between the target candidate and the suboptimal candidate set is flexibly aggregated to obtain the local soft margin score of the corresponding generation position, so that the local soft margin score simultaneously represents the leading degree of the target candidate and the competitive intensity formed by multiple suboptimal candidates against the target candidate. When the target output sequence contains multiple generation positions, the local soft margin scores corresponding to each generation position are subjected to sequence-level aggregation processing to obtain the sequence-level uncertainty quantification score corresponding to the target output sequence. The sequence-level aggregation processing is used to highlight the limiting effect of the generation position with the highest uncertainty in the target output sequence on the overall output credibility. When the target output sequence contains only one generation position, the local soft margin score corresponding to that generation position is used as the sequence-level uncertainty quantization score. The sequence-level uncertainty quantization score is compared with a preset security threshold, and differential processing is performed on the target output sequence based on the comparison result. When the sequence-level uncertainty quantization score is greater than or equal to the preset security threshold, the target output sequence is output and downstream business processing corresponding to the target output sequence is triggered. When the sequence-level uncertainty quantization score is less than the preset security threshold, direct acceptance of the target output sequence is blocked, and the request corresponding to the natural language input is routed to the safe fallback channel. The construction of the local candidate set, the determination of the local soft margin score, the generation of the sequence-level uncertainty quantization score, and the differential processing based on the sequence-level uncertainty quantization score are all related processing processes that are continuously executed for the same forward inference result. The process of obtaining the local soft margin score is always completed within the local candidate set based on the original Logit score vector. No probability normalization processing is performed on the local candidate set, and no multiple sampling and generation are performed for the same natural language input.

[0011] According to one embodiment of this application, the local candidate extraction process for the original Logit score vectors corresponding to each generation position includes: The scores corresponding to each candidate token in the original Logit score vector are sorted by size, and the top k candidate tokens are selected as the local candidate set, where k is a preset integer greater than 2.

[0012] According to one embodiment of this application, the target candidate is the candidate token ranked first in the local candidate set, and the suboptimal candidate set includes the remaining candidate tokens in the local candidate set other than the target candidate.

[0013] According to one embodiment of this application, the flexible aggregation processing of the comprehensive competitive relationship between the target candidate and the set of suboptimal candidates includes: The original Logit scores corresponding to each candidate token in the suboptimal candidate set are subjected to exponential transformation and summed, and then the summation result is subjected to logarithmic transformation to obtain the comprehensive competitive intensity corresponding to the suboptimal candidate set. The local soft margin score is determined based on the difference between the original Logit score of the target candidate and the overall competitiveness.

[0014] According to one embodiment of this application, the sequence-level aggregation processing of the local soft margin scores corresponding to each generated position includes: The minimum value among the local soft margin scores corresponding to each generation position is determined, and the minimum value is used as the sequence-level uncertainty quantization score.

[0015] According to one embodiment of this application, after extracting the original Logit score vector, the method further includes: When the large language model performs inference in low-precision floating-point format, the original Logit score vector is converted into single-precision floating-point format, and then the local candidate extraction process and the local soft margin score determination process are performed.

[0016] According to one embodiment of this application, the safe fallback channel includes a model cascade scheduling channel, which forwards the corresponding request to another large language model with a larger parameter scale than the large language model for secondary inference when the sequence-level uncertainty quantization score is less than the preset safety threshold. The preset security threshold is determined through offline verification, specifically including: Based on the verification dataset with correct annotations, the sequence-level uncertainty quantification score corresponding to each sample is calculated, and the correctness of the model output is determined in combination with the preset evaluation index to determine the preset safety threshold.

[0017] According to one embodiment of this application, the target output sequence includes a single token output sequence or a multi-token output sequence; In the scenario corresponding to the single token output sequence, the local soft margin score at the corresponding generation position is directly used as the sequence-level uncertainty quantization score.

[0018] According to one embodiment of this application, the downstream service processing includes a service interface call after intent recognition; When the sequence-level uncertainty quantization score is greater than or equal to the preset security threshold, the corresponding business interface is invoked according to the intent category represented by the target output sequence. When the sequence-level uncertainty quantization score is less than the preset security threshold, the business interface call is suspended.

[0019] A second aspect of this application provides a reliable processing system for the output of a large language model based on local soft margins, characterized in that it includes: The inference module receives natural language input and controls the large language model to perform a single forward inference to generate the target output sequence. The uncertainty quantization module is used to extract the original Logit score vector without Softmax normalization for each generation position during the generation process of the target output sequence, perform local candidate extraction processing on the original Logit score vector corresponding to each generation position, determine the target candidate and the suboptimal candidate set, determine the local soft margin score of the corresponding generation position based on the comprehensive competitive relationship between the target candidate and the suboptimal candidate set, and generate the sequence-level uncertainty quantization score based on the local soft margin score corresponding to each generation position. The threshold decision module is used to compare the sequence-level uncertainty quantization score with a preset safety threshold to obtain a release decision result or a backoff decision result; The routing execution module is used to output the target output sequence and trigger the corresponding downstream service processing when the release determination result is true, and to block the direct acceptance of the target output sequence when the fallback determination result is true, and to route the corresponding request to the safe fallback channel. The uncertainty quantification module is configured as follows: The original Logit scores corresponding to each candidate token in the suboptimal candidate set are subjected to exponential transformation and summed, and then the summation result is subjected to logarithmic transformation to obtain the comprehensive competitive intensity corresponding to the suboptimal candidate set. The local soft margin score is determined based on the difference between the original Logit score of the target candidate and the overall competitiveness.

[0020] Due to the adoption of the above technical solution, the beneficial effects achieved by this application are as follows: This application performs only one forward inference based on the same natural language input and does not perform multiple sampling and generation for the same natural language input. Therefore, it can complete uncertainty quantification under the condition of single inference, reduce computational overhead and inference latency, and is more suitable for high concurrency and low latency scenarios.

[0021] This application extracts the original Logit score vector without Softmax normalization during the large language model generation process, and directly obtains the local soft margin score based on the original Logit score vector within the local candidate set. Therefore, it can avoid the compression of absolute score difference information caused by probability normalization, which is beneficial to preserving the original competitive relationship and overall confidence level information between candidate tokens.

[0022] This application constructs a local candidate set by selecting a predetermined number of candidate tokens with the highest scores from all candidates. This allows the uncertainty quantification focus to be limited to the top candidates that are actually competitive with the current output, thereby reducing the interference of long-tail candidate noise on the quantization results.

[0023] This application does not only utilize the probability or score of the target candidate itself, nor only the difference between the top two candidates, but also performs flexible aggregation processing on the comprehensive competitive relationship between the target candidate and the suboptimal candidate set based on the original Logit score of the target candidate and the original Logit score of each candidate token in the suboptimal candidate set. This allows the local soft margin score to represent both the leading degree of the target candidate and the competitive intensity formed by multiple suboptimal candidates against the target candidate, thus making it more conducive to identifying the hesitant state of the model when multiple candidates compete simultaneously.

[0024] When the target output sequence contains multiple generation positions, this application performs sequence-level aggregation processing on the local soft margin scores corresponding to each generation position, and highlights the limiting effect of the generation position with the highest uncertainty on the overall output credibility. Therefore, it can effectively map the position-level uncertainty quantification result to the output-level uncertainty quantification result, and avoid the local high-risk generation position being masked by the overall aggregation result.

[0025] This application compares the sequence-level uncertainty quantification score with a preset security threshold, and performs release or security rollback processing on the target output sequence based on the comparison result. This allows the uncertainty quantification result to be directly used as a system-level decision-making basis, and timely blocks risky outputs from entering downstream business processes when the output credibility is insufficient, thereby improving the security and controllability of system operation. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a reliable processing method for large language model output based on local soft margins, provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Figure 3 A flowchart of the local soft margin uncertainty quantification method provided in the embodiments of this application; Figure 4 A schematic diagram of a trusted artificial intelligence system architecture provided in the embodiments of this application; Figure 5 A comparison of AUROC values ​​between the local soft margin method provided in this application embodiment and other white-box baseline methods on two datasets; Figure 6 A comparison of AUROC values ​​for the local soft margin method and the black-box multiple sampling method provided in this application on the SNIPS dataset.

[0027] Figure label: 810, Processor; 820, Communication interface; 830, Memory; 840, Communication bus. Detailed Implementation

[0028] To more clearly illustrate the overall concept of this application, a detailed explanation is provided below with reference to the accompanying drawings.

[0029] Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below. It should be noted that, unless otherwise specified, the embodiments of this application and the features thereof can be combined with each other.

[0030] In this application, unless otherwise expressly specified and limited, the "above" or "below" of the second feature can mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0031] Example 1 like Figure 1 As shown, a reliable processing method for the output of a large language model based on local soft margins includes: S1. Receive natural language input and control the large language model to perform a single forward inference based on the natural language input to generate the target output sequence.

[0032] Specifically, the natural language input refers to user queries, instructions, descriptions, or dialogue content expressed in human natural language. This input can originate from the interactive interface on the terminal device, application programming interface calls, or the text transmission interface of the business system. A large language model refers to a neural network model that has been pre-trained on a large-scale text corpus and possesses contextual understanding and text generation capabilities, such as an autoregressive language model based on the Transformer architecture.

[0033] Upon receiving the natural language input, this application controls the large language model to perform a single forward inference process. "Single forward inference" means that for a single natural language input, the large language model performs only one complete forward propagation computation, generating the target output sequence position-by-position through an autoregressive approach during this process. In other words, for the same natural language input, this application does not perform multiple repeated sampling and generation processes, nor does it rely on comparisons or consistency analyses between multiple output sequences.

[0034] By generating the target output sequence through a single forward inference operation, this application ensures output generation efficiency while avoiding the additional computational overhead and response latency caused by multiple sampling. This design allows subsequent uncertainty quantification and reliable decision-making processes to rely entirely on the intermediate state information generated during the same forward inference process, making it more suitable for high-concurrency, low-latency real-world business deployment scenarios.

[0035] The target output sequence refers to the token sequence corresponding to the final answer text generated by the large language model for natural language input. Depending on the specific task scenario, the target output sequence can be a sequence consisting of multiple generation positions, such as the text of an answer consisting of one or more sentences; or it can be a single token output sequence containing only one generation position, such as a single category label or intent identifier output in a classification task or intent recognition task. Regardless of the length of the target output sequence, this application performs uncertainty analysis on each generation position to comprehensively evaluate the overall credibility of the output.

[0036] S2. During the generation of the target output sequence, the original Logit score vector, which is not Softmax normalized, is extracted from the output of the large language model at each generation position.

[0037] Specifically, when the large language model generates the target output sequence through autoregression, for each generation position (i.e., each token position to be generated), the model's output layer calculates a vector with dimensions equal to the vocabulary size. Each element in this vector corresponds to the original score of the corresponding candidate token in the vocabulary, i.e., the Logit score. In the conventional token selection process, this original Logit score vector usually needs to be normalized using the Softmax function to convert it into a probability distribution vector, and then sampling or greedy selection is performed based on the probability values.

[0038] This application extracts the intermediate information required for uncertainty quantification by directly extracting the original Logit score vector without Softmax normalization, instead of using normalized probability values. The purpose of this is that the Softmax operation compresses the absolute differences between original scores into relative proportions. Especially when the score range is large, the normalized probability values ​​may not accurately reflect the original competitive intensity between candidate tokens. For example, when the original Logit scores of the target candidate and the second-best candidate differ significantly, the probability difference after Softmax transformation may be compressed to a very small range, thereby reducing the discriminative power of the uncertainty quantification results.

[0039] Therefore, this application extracts the original Logit score vector at each generation position during the generation process and uses it as the basis for subsequent local candidate extraction and local soft margin score calculation. It should be noted that this extraction operation is completed naturally during a single forward inference iteration of the large language model, without requiring additional model calls or secondary calculations; only the Logit output at the corresponding position needs to be retained or transferred in the model inference interface. By preserving the score information in the original Logit space, this application can more sensitively capture the model's absolute preference for each candidate token in the current context, thus providing an accurate data foundation for constructing highly reliable uncertainty quantification indicators.

[0040] S3. Perform local candidate extraction processing on the original Logit score vector corresponding to each generation position to select a preset number of candidate tokens with the highest scores from all candidates corresponding to the vocabulary, forming a local candidate set for the corresponding generation position, and determine the target candidate ranked first and the set of second-best candidates other than the target candidate in the local candidate set.

[0041] Specifically, because the vocabulary of large language models is typically enormous, ranging from tens of thousands to hundreds of thousands of words, the original Logit score vector contains a large number of long-tail candidate tokens that have extremely weak semantic relevance to the current generation position and have a negligible probability of being selected. These long-tail candidates not only increase the computational burden, but their score fluctuations are often caused by randomness during model training, and therefore have no substantial reference value for reflecting the model's confidence level at the current position. If analysis is performed directly based on the entire vocabulary, long-tail noise will interfere with the accuracy and stability of the uncertainty quantification results.

[0042] To address this, this application sorts the scores of each candidate token in the original Logit score vector in descending order for each generation location, and extracts a predetermined number (denoted as k) of the top-ranked candidate tokens. These k candidate tokens constitute the local candidate set corresponding to that generation location. Here, k is a pre-defined integer, and k is greater than 2. For example, in one embodiment of this application, k can be 5, 10, or 20, and the specific value can be determined based on the trade-off between computational efficiency and evaluation accuracy in the actual deployment scenario. Choosing a k value that is too small may lead to the omission of some competitive suboptimal candidates; choosing a k value that is too large will introduce unnecessary long-tail interference and increase computational overhead.

[0043] After constructing a local candidate set, this application further identifies two types of roles within that set: The first is the target candidate, which is the candidate token ranked first in the original Logit score in the local candidate set. This candidate token is the result that the model is most likely to output under the standard generation process. The second is the suboptimal candidate set, which consists of the remaining k-1 candidate tokens from the local candidate set, excluding the target candidate. These tokens represent other candidates that have a potential competitive relationship with the target candidate at the current generation position.

[0044] Through the aforementioned local candidate extraction process, this application compresses the scope of uncertainty quantification analysis from the large and noisy full vocabulary space to a head candidate subspace with practical competitive significance. This process brings two significant benefits: First, it effectively suppresses the interference of score noise from long-tail candidates in the vocabulary on subsequent uncertainty quantification, making the evaluation metric more focused on the model's hesitation among the main candidates; second, while retaining sufficient competitive information, it significantly reduces the computational complexity of subsequent flexible aggregation processing, which is beneficial for maintaining low latency in online inference scenarios.

[0045] S4. Based on the original Logit score of the target candidate and the original Logit score of each candidate token in the suboptimal candidate set, perform flexible aggregation processing on the comprehensive competitive relationship of the target candidate relative to the suboptimal candidate set to obtain the local soft margin score of the corresponding generation position, so that the local soft margin score simultaneously represents the leading degree of the target candidate and the competitive intensity formed by multiple suboptimal candidates against the target candidate.

[0046] Specifically, common uncertainty measurement methods in existing technologies have limitations when dealing with relationships between multiple candidates. For example, methods that only consider the score difference between the target candidate and the second-ranked candidate (i.e., the hard margin) ignore the possibility that the third-ranked and subsequent candidates may also have high scores. When the model hesitates among multiple candidates, or when multiple suboptimal candidates are highly competitive, the difference between the top two candidates alone is insufficient to fully reflect the true uncertainty level of the current generated position.

[0047] To overcome the aforementioned limitations, this application introduces a "flexible aggregation processing" mechanism. This processing does not simply compare second-best candidates as isolated individuals, but rather treats the entire set of second-best candidates as a whole, evaluating the overall competitive strength they exert on the target candidate. The "flexibility" here is reflected in two aspects: first, the aggregation process automatically adapts to differences in score distribution within the second-best candidate set, effectively highlighting the contribution of high-scoring candidates to the overall competitive strength while naturally weakening the influence of low-scoring candidates; second, this aggregation is not a rigid numerical superposition, but rather a smooth mapping of the score space through a logarithmic-exponential transformation, thereby obtaining a more discriminative characterization of competitive strength.

[0048] In one embodiment of this application, the flexible polymerization process is specifically implemented by including the following steps: First, an exponential transformation is performed on the original Logit score corresponding to each candidate token in the suboptimal candidate set. The purpose of the exponential transformation is to map the linear score in the Logit space to the exponential domain, thereby giving higher-scoring candidates a greater weight contribution in the subsequent summation process, naturally achieving the effect of "highlighting the strong and suppressing the weak".

[0049] Secondly, the scores of all suboptimal candidates after exponential transformation are summed to obtain a scalar value. This summation result can be regarded as an exponential domain representation of the "overall competitiveness" formed by the entire set of suboptimal candidates at the current generation position.

[0050] Next, a logarithmic transformation is performed on the above summation result to map the aggregation result from the exponential domain back to the logarithmic domain, which is comparable to the original Logit score, to obtain the comprehensive competitive strength corresponding to the suboptimal candidate set.

[0051] Based on this, this application determines the local soft margin score corresponding to the generated position by calculating the difference between the original Logit score of the target candidate and the overall competitive intensity. This difference value intuitively reflects the extent to which the target candidate "leads" the overall competitive force formed by all major competitors. When the advantage of the target candidate is obvious, the local soft margin score is a large positive value; when the overall competitiveness of the second-best candidate set is strong or even comparable to that of the target candidate, the local soft margin score tends to be close to zero or negative.

[0052] Through the aforementioned flexible aggregation process, the local soft margin score obtained in this application can simultaneously characterize two aspects of information: first, the leading position of the target candidate, i.e., the size of the target candidate's advantage in score relative to the competing set; second, the competitive intensity formed by multiple suboptimal candidates against the target candidate, i.e., the concentration of score distribution within the competing set and the overall energy level. Compared to traditional single difference or full vocabulary entropy values, this local soft margin score, while retaining computational simplicity, can more delicately and robustly characterize the confidence state of the model at a single token generation position, providing a high-quality basic unit for subsequent sequence-level uncertainty quantification.

[0053] It is important to note that throughout the entire flexible aggregation process, this application always completes the process based on the original Logit score vector within the local candidate set. It neither re-performs Softmax probability normalization on the scores of the local candidate set nor relies on any additional sampling or auxiliary models. This design ensures the fidelity of the uncertainty quantification process and the original model output information, avoids information compression caused by normalization, and avoids the computational overhead of multiple sampling. Therefore, this application's solution can efficiently and accurately complete the position-level uncertainty measurement under single inference conditions.

[0054] S5. When the target output sequence contains multiple generation positions, perform sequence-level aggregation processing on the local soft margin scores corresponding to each generation position to obtain the sequence-level uncertainty quantification score corresponding to the target output sequence. The sequence-level aggregation processing is used to highlight the limiting effect of the generation position with the highest uncertainty in the target output sequence on the overall output credibility.

[0055] Specifically, the target output sequence generated by autoregressive large language models is typically composed of multiple generation positions sequentially. Each generation position has its corresponding local soft margin score, which reflects the model's local confidence level when generating that specific token. However, low uncertainty at a single position is insufficient to guarantee the reliability of the entire output sequence. This is because, under the autoregressive generation mechanism, a generation error at any position in the output sequence can be propagated through context, affecting the generation quality of subsequent positions and even causing the entire output sequence to deviate from the user's intent or factual basis. Therefore, it is necessary to integrate the local uncertainty indices of multiple positions into a sequence-level score that can represent the credibility of the entire sentence output.

[0056] This application follows an important principle in sequence-level aggregation processing: highlighting the limiting effect of the generation position with the highest uncertainty in the target output sequence on the overall output credibility. Intuitively, this principle means that the overall credibility of the target output sequence is constrained by its weakest link. If a position in the sequence exhibits significant uncertainty or hesitation (i.e., a low local soft margin score), even if other positions have sufficient confidence, this weak position may still lead to factual errors or logical breaks, thus rendering the entire output unreliable.

[0057] To achieve the above aggregation principle, in one embodiment of this application, the sequence-level aggregation process specifically includes: determining the minimum value among the local soft margin scores corresponding to each generation position, and using the minimum value as the sequence-level uncertainty quantization score.

[0058] The advantages of using the minimum value aggregation method are: First, it naturally satisfies the requirement of characterizing the "barrel effect," that is, the overall confidence upper limit of a sequence is determined by its lowest confidence position. The sequence-level uncertainty quantification score can only show a high value if and only if the local soft margin scores of all generation positions are maintained at a high level, indicating that the model has a stable and sufficient grasp of the entire sentence output.

[0059] Second, the minimum value aggregation method has the advantages of simple calculation and no need to introduce additional hyperparameters. In the inference stage, it is only necessary to maintain a rolling minimum value during the generation process, which hardly increases the latency overhead.

[0060] Third, minimum value aggregation is insensitive to changes in sequence length. Compared to aggregation methods such as summation or averaging, minimum value aggregation avoids the defect of masking individual high-risk positions due to long sequences. For example, in a long output sequence, even if most positions have high scores, minimum value aggregation can still accurately mark the sequence as high-risk as long as there is one position with an extremely low score.

[0061] Through the aforementioned sequence-level aggregation process, this application effectively maps the locally soft margin scores obtained from independent evaluations of each generation position into a quantitative index of overall uncertainty for the complete target output sequence. This index can keenly identify output results with local generation risks and thus questionable overall credibility, providing a reliable basis for subsequent threshold decisions and differentiated processing.

[0062] S6. When the target output sequence contains only one generation position, the local soft margin score corresponding to the generation position is used as the sequence-level uncertainty quantization score.

[0063] Specifically, the output format of large language models varies across different application scenarios. Besides free text generation tasks that generate multiple token sequences, there are also task scenarios where the output format is a single token sequence. Typical examples include: user intent classification tasks, where the model outputs a single category label token; sentiment analysis tasks, where the model outputs a single token corresponding to "positive" or "negative"; or in some question-answering scenarios, where the model only needs to output a single entity name token as the answer. In these tasks, the target output sequence has a length of 1, meaning it contains only one generation position.

[0064] In scenarios with a single generation location, the uncertainty of the output sequence is essentially equivalent to the uncertainty of that single generation location. There is no aggregation problem between multiple locations, nor is it necessary to consider the mutual constraints between the uncertainties of different locations. Therefore, when determining that the target output sequence contains only one generation location, this application directly assigns the local soft margin score of that generation location, calculated through the aforementioned steps, as the sequence-level uncertainty quantization score.

[0065] It should be noted that the method in this application already covers the differences in handling single-token and multi-token output scenarios in its process design. For multi-token output sequences, as mentioned earlier, this application adopts sequence-level aggregation processing based on the minimum value to highlight the limiting effect of weak links; while for single-token output sequences, since there is no weak link propagation effect within the sequence, no additional aggregation operation is required, and the local soft margin score is directly used as the uncertainty assessment result at the output level. This processing method is logically consistent and maintains consistency with the evaluation framework of multi-token scenarios, that is, both use the most uncertain point exhibited by the model during the generation process as the benchmark for overall credibility. In the single-token scenario, the most uncertain point is naturally the unique generation position itself.

[0066] By handling the above-mentioned different scenarios, this application ensures the applicability and computational efficiency of the uncertainty quantification method under various task forms, so that whether it is a short output task such as classification and label prediction or a long output task such as free text generation, a sequence-level uncertainty quantification score that accurately reflects the credibility of the model output can be obtained.

[0067] S7. Compare the sequence-level uncertainty quantization score with a preset security threshold, and perform differentiated processing on the target output sequence according to the comparison result. When the sequence-level uncertainty quantization score is greater than or equal to the preset security threshold, output the target output sequence and trigger downstream business processing corresponding to the target output sequence. When the sequence-level uncertainty quantization score is less than the preset security threshold, block the direct acceptance of the target output sequence and route the request corresponding to the natural language input to the secure fallback channel.

[0068] Specifically, after obtaining the sequence-level uncertainty quantization score corresponding to the target output sequence, this application compares the score with a pre-set preset security threshold and performs differential processing on the target output sequence based on the comparison result.

[0069] The preset safety threshold is a threshold used to distinguish the level of confidence in the model output. This threshold is not subjectively set, but rather obtained through statistical analysis and calibration of the validation dataset during the offline phase. Specifically, based on the correctly labeled validation dataset, the sequence-level uncertainty quantification score corresponding to each sample is calculated, and combined with the binary classification label indicating whether the model output is correct, the preset safety threshold is determined according to a preset evaluation metric (e.g., a threshold value that guarantees a specific recall or precision). In this way, the preset safety threshold can be solidified into a configuration parameter that matches the risk preferences of the specific application scenario before actual deployment.

[0070] Based on the comparison results, this application adopts two different processing paths: Firstly, when the sequence-level uncertainty quantification score is greater than or equal to the preset security threshold, it indicates that the model has a high degree of confidence in the currently generated output sequence, and the output content is credible and sufficient to be directly accepted by the business system. In this case, this application outputs the target output sequence, that is, the answer text or category label generated by the large language model is returned to the caller normally, and the downstream business processing corresponding to the target output sequence is triggered simultaneously. The specific form of the downstream business processing depends on the application scenario. For example, in the intent recognition task, the corresponding business interface is called according to the intent category represented by the target output sequence to perform subsequent operations; in the intelligent question answering system, the generated answer is directly pushed to the user terminal; in the knowledge extraction scenario, the extracted structured information is written into the database or the subsequent workflow is started.

[0071] Secondly, when the sequence-level uncertainty quantification score is less than the preset security threshold, it indicates that the model has significant uncertainty or hesitation during the generation process, and the output content has a high risk of error and is not suitable for direct acceptance. In this case, this application blocks the direct acceptance of the target output sequence, that is, it does not return the output sequence as a reliable result to the downstream business module, thereby preventing unreliable content from flowing into the business process and causing erroneous decisions or security incidents. At the same time, this application routes the request corresponding to the natural language input to a secure fallback channel.

[0072] The safety fallback channel is a pre-configured alternative processing path used to provide a safety net when the model output lacks credibility. In one embodiment of this application, the safety fallback channel can be a model cascading scheduling channel, that is, when the output of the current large language model is found to be uncertain, the same natural language input request is forwarded to another large language model with a larger parameter scale, stronger performance, but higher inference cost for secondary inference, in order to obtain a more reliable output result. In other embodiments, the safety fallback channel can also be set as a manual review queue, a templated response mechanism, or an interactive process that prompts the user to rephrase the input. The specific choice depends on the comprehensive trade-off between accuracy, latency, and resource cost in the business scenario.

[0073] Through the aforementioned threshold comparison and differentiation processing mechanism, this application transforms the uncertainty quantification score into a decision-making basis that can directly guide system behavior. On the one hand, for high-confidence outputs, the system maintains efficient flow, fully utilizing the capabilities of the large language model to support business automation; on the other hand, for low-confidence, high-risk outputs, the system automatically triggers a security protection mechanism to prevent erroneous content from entering downstream processes. This mechanism effectively balances the efficiency and reliability of model application, significantly improving the robustness and security of business systems based on large language models in actual operation.

[0074] The construction of the local candidate set, the determination of the local soft margin score, the generation of the sequence-level uncertainty quantization score, and the differential processing based on the sequence-level uncertainty quantization score are all related processing processes that are continuously executed for the same forward inference result. The process of obtaining the local soft margin score is always completed within the local candidate set based on the original Logit score vector. No probability normalization processing is performed on the local candidate set, and no multiple sampling and generation are performed for the same natural language input.

[0075] Specifically, the construction of the local candidate set, the determination of the local soft margin score, the generation of the sequence-level uncertainty quantification score, and the differential processing based on the sequence-level uncertainty quantification score constitute a sequentially executed associative processing procedure for the same forward inference result. This overall process design has the following three explicit constraints and characteristics: First, the entire process is completed based on a single forward inference. This application does not perform multiple sampling and generation for the same natural language input. In other words, after receiving a natural language input, the large language model performs only one complete round of forward propagation and autoregressive generation, producing a unique target output sequence. All subsequent uncertainty quantification steps—from extracting the original Logit score vectors at each generation position, constructing local candidate sets, calculating local soft margin scores, to aggregating the generated sequence-level uncertainty quantification scores, and finally to threshold comparison and differential routing decisions—rely on intermediate state information generated or readily available during this forward inference process. This design fundamentally avoids the problem of increased inference latency and computational power consumption caused by multiple independent sampling or multiple forward computations in existing black-box methods, enabling the proposed solution to be seamlessly embedded in high-concurrency, low-latency online service systems.

[0076] Second, computation is always performed within the original Logit space and the local candidate set. When obtaining the local soft margin scores for each generation position, this application always uses the original Logit score vector without Softmax normalization, and after constructing the local candidate set, computation is performed only within the scope of the local candidate set. During this process, no probability normalization is performed on the local candidate set. Specifically, this application does not re-perform the Softmax transformation on the scores of the first k extracted candidate tokens to convert them into a local probability distribution. This is because, while re-performing local Softmax can make the candidate scores meet the normalization constraints of the probability distribution, it also compresses the absolute difference information between the original Logit scores, distorting the numerical differences reflecting the model's absolute preference into values ​​that only reflect local relative proportions, thereby weakening the uncertainty index's ability to represent the true confidence level. This application directly performs flexible aggregation processing based on the original Logit scores, preserving the true score relationship of candidate tokens in the uncompressed state of the model's output layer, allowing the local soft margin scores to more faithfully reflect the model's original judgment strength and degree of hesitation at the current position.

[0077] Third, the processing steps are continuous and interconnected. From the construction of the local candidate set to the final differential processing, the various steps in this application are not isolated but constitute a tightly connected processing chain. The output of the upstream step serves as the direct input of the downstream step, and the entire processing is completed sequentially within the natural time window of a single inference, without introducing additional asynchronous waiting or external service calls. This continuously executed, interconnected processing process ensures that the uncertainty quantification results are synchronized with the model generation process in real time, enabling the system to complete the assessment and handling decision of the output credibility before the target output sequence is finally returned to the user or caller, thereby achieving true online risk management.

[0078] By adhering to four core design constraints—"single-inference," "original Logit space," "no local renormalization," and "continuous association processing"—this application minimizes the impact on inference efficiency while ensuring the accuracy and sensitivity of uncertainty quantification. It provides a technical solution that balances real-time performance, computational economy, and evaluation effectiveness, offering solid support for the reliable deployment of large language models in critical business scenarios.

[0079] According to one embodiment of this application, the local candidate extraction process for the original Logit score vectors corresponding to each generation position includes: The scores corresponding to each candidate token in the original Logit score vector are sorted by size, and the top k candidate tokens are selected as the local candidate set, where k is a preset integer greater than 2.

[0080] Specifically, after obtaining the original Logit score vector for a generation position, this vector contains the scores of all candidate tokens in the vocabulary, and its dimension is usually consistent with the vocabulary size. To construct a local candidate set, this application first sorts the scores corresponding to each candidate token in the vector in descending order, i.e., the highest score is placed first, the second highest score is placed second, and so on. After sorting, this application extracts the candidate tokens ranked in the top k positions and includes the identifiers of these k candidate tokens and their corresponding original Logit scores into the local candidate set for that generation position.

[0081] The value of parameter k has specific technical significance in this application. First, k is set to a preset integer greater than 2. The reason for requiring k to be greater than 2 is that one of the technical goals of this application is to overcome the limitation of traditional methods that only observe the top two candidates (i.e., k=2). When k is only 2, the local candidate set only contains the target candidate and the second-ranked suboptimal candidate. At this time, the evaluation of the competitive relationship degenerates into the traditional hard margin difference calculation, which cannot reflect the hesitation state of the model when multiple suboptimal candidates have high scores at the same time. By setting k to a value greater than 2, the local candidate set can accommodate more candidate tokens with substantial competitive significance, thereby providing the necessary information basis for subsequent flexible aggregation processing, enabling the local soft margin score to comprehensively perceive the competitive pressure formed by multiple suboptimal candidates.

[0082] Secondly, the specific value of k can be preset and adjusted according to the actual deployment scenario. In different embodiments of this application, k can be 5, 10, 20, or other reasonable values. The choice of k value reflects the trade-off between "information sufficiency" and "computational overhead" in uncertainty assessment: the larger the k value, the more comprehensive the competitive candidates covered by the local candidate set, and the tolerance of the assessment results to long-tail noise may be correspondingly improved, but at the same time, the computational load of sorting and aggregation operations also increases; the smaller the k value, the higher the computational efficiency, but some candidates with a certain degree of competitiveness may be missed, resulting in an underestimation of uncertainty. In practical applications, the value of k can be calibrated through offline verification datasets, and a suitable value that balances accuracy and efficiency can be determined based on preset uncertainty detection performance indicators.

[0083] Through the aforementioned sorting and truncation operations, this application compresses the analysis scope from a large and noisy full vocabulary space to a local candidate set space of size k. This process significantly reduces the computational complexity of subsequent flexible aggregation processing while ensuring the effective preservation of competitive information, and also avoids the statistical interference caused by low-scoring long-tail candidates to the uncertainty quantification results.

[0084] According to one embodiment of this application, the target candidate is the candidate token ranked first in the local candidate set, and the suboptimal candidate set includes the remaining candidate tokens in the local candidate set other than the target candidate.

[0085] Specifically, after the local candidate set is constructed, this application further clarifies the role classification of each candidate token. Specifically, the target candidate is the candidate token ranked first in the original Logit score of the local candidate set, and the suboptimal candidate set includes the remaining candidate tokens in the local candidate set other than the target candidate.

[0086] The aforementioned role division has a clear logical basis and subsequent processing purpose. The local candidate set consists of the top k candidate tokens extracted from the full vocabulary candidate list, where the highest-scoring candidate token represents the preferred prediction result calculated based on the generated context and model parameter weights at the current generation position. In a conventional autoregressive generation process, this candidate token is the output content that will be actually selected under the greedy decoding strategy. Therefore, this application defines the top-ranked candidate token as the target candidate and regards it as a reference benchmark for evaluating the uncertainty of the current position.

[0087] Correspondingly, the remaining k-1 candidate tokens in the local candidate set, excluding the target candidate, are collectively categorized into the suboptimal candidate set. Although these candidate tokens are not the model's first choice at the current generation position, they all possess a certain score level and competitiveness, representing alternatives that the model considers "possible" in the current context. When multiple candidate tokens in the suboptimal candidate set have high scores and the score difference with the target candidate is small, it indicates that the model hesitates in multiple semantic directions, and the uncertainty of the current position is high. Conversely, if the scores in the suboptimal candidate set are generally low and the score difference with the target candidate is significant, it indicates that the model has an overwhelming preference for the target candidate, and the uncertainty of the current position is low.

[0088] By explicitly dividing the local candidate set into two parts—the target candidate set and the suboptimal candidate set—this application constructs a clear binary analysis framework for subsequent flexible aggregation processing. Within this framework, the score of the target candidate serves as the "leading" indicator being compared, while the suboptimal candidate set is considered as the "competitors" as a whole requiring comprehensive evaluation. This division allows the focus of uncertainty quantification to be concentrated on the relationship between the target candidate and its main competitors. This avoids the noise interference from full-vocabulary analysis and captures the complex uncertainty patterns when multiple candidates compete, compared to a simple comparison of only the top two candidates.

[0089] According to one embodiment of this application, the flexible aggregation processing of the comprehensive competitive relationship between the target candidate and the set of suboptimal candidates includes: The original Logit scores corresponding to each candidate token in the suboptimal candidate set are subjected to exponential transformation and summed, and then the summation result is subjected to logarithmic transformation to obtain the comprehensive competitive intensity corresponding to the suboptimal candidate set. The local soft margin score is determined based on the difference between the original Logit score of the target candidate and the overall competitiveness.

[0090] Specifically, an exponential transformation is performed on the original Logit scores corresponding to each candidate token in the suboptimal candidate set, and the scores are summed. Then, a logarithmic transformation is performed on the summed scores to obtain the overall competitive intensity corresponding to the suboptimal candidate set. Next, based on the difference between the original Logit scores of the target candidate and the overall competitive intensity, the local soft margin score corresponding to the generation position is determined.

[0091] The mathematical essence of the above processing can be understood as a smooth maximum aggregation operation on the suboptimal candidate set in the original Logit space. The technical meaning and design considerations of each step are explained in detail below.

[0092] Regarding exponential transformation and summation: An exponential transformation is performed on the original Logit score of each candidate token in the suboptimal candidate set, mapping the score to its natural exponential function value. Mathematically, the exponential transformation exhibits non-linear amplification: for larger input values, the output value is significantly amplified; for smaller input values, the output value is compressed to near zero. Therefore, when summing the exponential transformation values ​​of each candidate in the suboptimal candidate set, candidate tokens with higher scores will make a dominant contribution to the summation result, while the contribution of candidate tokens with lower scores is naturally suppressed. This mechanism achieves "flexible weighting" of scores within the suboptimal candidate set, automatically highlighting highly competitive candidates and weakening less competitive candidates without explicitly setting weight coefficients. This allows the aggregation result to centrally reflect the main part of the suboptimal candidate set that truly poses competitive pressure to the target candidate.

[0093] Regarding the logarithmic transformation: After performing exponential summation, a logarithmic transformation is performed on the summation result, mapping the aggregated value from the exponential domain back to the same logarithmic domain space as the original Logit score. This inverse transformation has a dual function: first, it restores a numerical scale comparable to the original Logit score, facilitating subsequent comparisons with the target candidate score; second, the logarithmic transformation and the aforementioned exponential transformation are inverse operations in terms of their functional relationship, and their combination constitutes the mathematically known LogSumExp operator. This operator possesses good numerical stability and smooth differentiability, and can incorporate the contribution of the second largest value while preserving the maximum value information, achieving a "flexible maximum" estimation of a set of values.

[0094] Through the combined operations of exponential transformation, summation, and logarithmic transformation described above, this application obtains the comprehensive competitive intensity corresponding to the set of suboptimal candidates. This comprehensive competitive intensity can be intuitively understood as the Logit score of a virtual "representative competitor," which aggregates the competitive energy of all suboptimal candidates, and its score level reflects the competitive pressure exerted by the set of suboptimal candidates as a whole on the target candidate.

[0095] Based on this, this application obtains the local soft margin score of the corresponding generation position by calculating the difference between the original Logit score of the target candidate and the overall competitive intensity. Specifically, this difference can be the target candidate score minus the difference in overall competitive intensity. When the target candidate score is significantly higher than the overall competitive intensity, the difference is a large positive value, indicating that the target candidate has a clear leading advantage at the current position, the model's selection tendency is firm, and the uncertainty is low; when the overall competitive intensity is close to or even exceeds the target candidate score, the difference approaches zero or becomes negative, indicating that the overall competitiveness of the suboptimal candidate set is comparable to or stronger than that of the target candidate, the model has obvious hesitation or wavering at the current position, and the uncertainty is high.

[0096] It is important to reiterate that all the above calculations are always performed based on the original Logit scores within the local candidate set. This application does not re-perform Softmax probability normalization on the scores of the local candidate set. This is because, once local normalization is introduced, the absolute differences between the original scores will be compressed into relative proportions, causing the subsequently calculated marginal scores to only reflect local relative relationships, rather than the original absolute preference strength of the model's output layer. This application directly performs flexible aggregation based on the original Logit space, preserving the true score information of the model in the uncompressed state of the output layer. This allows the local soft marginal scores to more accurately reflect the model's intrinsic confidence level, providing a more reliable numerical basis for uncertainty quantification.

[0097] According to one embodiment of this application, the sequence-level aggregation processing of the local soft margin scores corresponding to each generated position includes: The minimum value among the local soft margin scores corresponding to each generation position is determined, and the minimum value is used as the sequence-level uncertainty quantization score.

[0098] Specifically, in the autoregressive generation process of the target output sequence, for each generation position, this application calculates a local soft margin score through the aforementioned steps. This score reflects the local confidence level and candidate competition situation of the model when generating that specific token. When the target output sequence contains multiple generation positions, a set of local soft margin score sequences corresponding to the generation order is formed. The purpose of sequence-level aggregation processing is to integrate this set of position-level local uncertainty measures into a single comprehensive score that can characterize the credibility of the entire target output sequence.

[0099] This application employs a minimum value approach to achieve the aforementioned aggregation. Its technical rationale stems from the inherent risk characteristics of the autoregressive language model generation mechanism. In the autoregressive generation process, the prediction of subsequent tokens depends on the contextual information formed by previously generated tokens. If the output at a certain generation position in the sequence contains factual bias or logical breaks, even if the generation quality at other positions is high, this error may still be propagated and amplified as the generation process progresses, ultimately rendering the entire output sequence unreliable. In other words, the overall credibility of the target output sequence is constrained by its weakest link, following the "barrel effect" principle.

[0100] Using the minimum local soft margin score at each generation position in the sequence as the sequence-level uncertainty quantification score can accurately capture the aforementioned risk characteristics. If any generation position in the sequence exhibits a low uncertainty quantification value (i.e., a small local soft margin score), regardless of how high the scores at other positions are, the minimum value operation will drag the sequence-level score down to the level of that weak position, thus accurately labeling the output sequence as high-risk or low-confidence. Conversely, only when the local soft margin scores at all generation positions in the sequence remain at a high level will the sequence-level score exhibit a high value, indicating that the model has a continuous and sufficient grasp of the entire output sequence generation process.

[0101] Using the minimum aggregation method also has the following advantages at the engineering implementation level: First, the computational overhead is extremely low. During the position-by-position generation of the target output sequence, this application only needs to maintain a rolling minimum value variable. After calculating the local soft margin fraction for each position, a comparison operation is performed, and the smaller value is retained. This process does not involve complex floating-point operations or matrix operations, and adds almost no inference latency.

[0102] Second, it is insensitive to changes in sequence length. Unlike aggregation methods such as summation and averaging, minimum aggregation does not dilute the contribution of individual high-risk positions to the overall score due to a longer output sequence. Regardless of how many generating positions the sequence contains, as long as there is one position with high uncertainty, the sequence-level score can accurately reflect that risk.

[0103] Third, no additional hyperparameters are required. Minimum aggregation is a parameterless deterministic operation, avoiding the parameter tuning burden and scenario adaptation uncertainty caused by introducing hyperparameters such as weight coefficients and decay factors.

[0104] Through the above sequence-level aggregation process, this application effectively maps the location-level uncertainty assessment results into sequence-level credibility metrics, providing a concise, sensitive, and robust basis for subsequent threshold comparisons and differentiated routing decisions.

[0105] According to one embodiment of this application, after extracting the original Logit score vector, the method further includes: When the large language model performs inference in low-precision floating-point format, the original Logit score vector is converted into single-precision floating-point format, and then the local candidate extraction process and the local soft margin score determination process are performed.

[0106] Specifically, in practical engineering deployments, to reduce GPU memory usage and improve inference throughput, large language models often employ low-precision floating-point formats for forward computation, such as half-precision floating-point (FP16) or brain-like floating-point (BF16). Under these low-precision formats, the intermediate layer outputs and the final Logit score vector are stored and transmitted with lower numerical precision. While low-precision inference has a limited impact on the quality of most generation tasks, for the uncertainty quantification processing flow involved in this application, low-precision numerical representation may introduce potential risks.

[0107] In determining the local soft margin score, this application involves performing exponential transformation, summation, and logarithmic transformation on the original Logit scores of each candidate token in the suboptimal candidate set. These mathematical operations are sensitive to the dynamic range and precision of the input values. Under low-precision floating-point format, the following two types of problems may occur: First, there is the risk of numerical overflow. The exponential transformation operation (i.e., calculating the natural exponential function) can cause large input values ​​to grow rapidly. When the original Logit score is high, calculating the exponent in FP16 format may cause the result to exceed the maximum numerical range that the format can represent, resulting in overflow. This can cause the calculation to be interrupted or return an infinite value, thus compromising the correctness of subsequent aggregation processes.

[0108] Secondly, there is the issue of numerical underflow and precision loss. For negative scores with large absolute values, the exponential transformation result approaches zero. In low-precision format, these tiny values ​​may be truncated to exact zero, resulting in information loss. Furthermore, when summing multiple exponential transformation results, the limited number of decimal places in low-precision format may cause small values ​​to be completely submerged by large values, failing to effectively reflect the contribution of medium-competitive candidate tokens, thus affecting the accuracy and discriminative power of the overall competitiveness calculation.

[0109] To address the aforementioned issues, this application adds a format conversion step after extracting the original Logit score vector. When it is detected that the large language model is performing inference in low-precision floating-point format, the original Logit score vector is first converted to single-precision floating-point format (FP32). Then, based on the converted score vector, subsequent local candidate extraction processing and local soft-margin score determination processing are performed. Single-precision floating-point format has a larger exponent range and higher mantissa precision, which can provide sufficient numerical dynamic range and computational resolution for operations such as exponential transformation, summation, and logarithmic transformation. It effectively avoids the problems of overflow, underflow, and rounding error accumulation under low-precision format, ensuring the numerical stability and reliability of the local soft-margin score calculation process.

[0110] It should be noted that this format conversion operation is performed only within the uncertainty quantization module. The converted FP32 vector is only used for calculating local soft margin scores and is not fed back to the main inference process of the large language model. Therefore, it will not affect the model's inference accuracy or memory usage strategy. Furthermore, since the conversion object is only the Logit vector at the vocabulary dimension, its size is negligible relative to the overall model parameters, so the additional computational overhead caused by the format conversion is negligible. Through this lightweight accuracy enhancement, this application ensures the robustness and accuracy of the uncertainty quantization results while maintaining the efficiency advantages of deploying low-precision models.

[0111] According to one embodiment of this application, the safe fallback channel includes a model cascade scheduling channel, which forwards the corresponding request to another large language model with a larger parameter scale than the large language model for secondary inference when the sequence-level uncertainty quantization score is less than the preset safety threshold. The preset security threshold is determined through offline verification, specifically including: Based on the verification dataset with correct annotations, the sequence-level uncertainty quantification score corresponding to each sample is calculated, and the correctness of the model output is determined in combination with the preset evaluation index to determine the preset safety threshold.

[0112] Specifically, model cascading scheduling is a progressive computing resource scheduling strategy. In actual deployments, to balance inference efficiency and output quality, a large language model with a relatively small parameter size and fast inference speed is usually deployed at the front end as the main service model to handle the majority of daily requests. However, when faced with complex semantics, rare knowledge, or fuzzy input, the output uncertainty of the small-scale model is often high, and the risk of errors increases accordingly. If the small model continues to be relied upon for these requests identified as "low confidence," it may adversely affect downstream business.

[0113] This application provides an escalation path for such low-confidence requests through a model cascade scheduling channel. When the sequence-level uncertainty quantization score is lower than a preset safety threshold, it indicates that the current main model has doubts about the generated result of the request and should not be directly accepted. In this case, the system does not simply return a rejection or failure response, but forwards the request along with the obtained context information to another large language model with a larger parameter scale than the current large language model. This large language model with a large parameter scale usually has stronger semantic understanding capabilities, richer knowledge reserves, and higher generation accuracy, and can perform secondary inference on the same natural language input and generate more reliable output results.

[0114] Using a larger model as the target for safety fallback has several advantages: First, larger models often exhibit higher robustness and accuracy when handling highly uncertain inputs, effectively compensating for the limitations of smaller models in complex scenarios. Second, by only invoking the larger model when necessary, the high inference costs and response latency associated with directing all traffic to it are avoided, achieving cost optimization while ensuring service quality. This cascaded architecture of "small model for routine service, large model as a safety net" demonstrates good scalability and economy in engineering practice.

[0115] Furthermore, this application embodiment also clarifies the method for determining the preset security threshold. The preset security threshold is not set by human experience, but is systematically determined through offline verification. The specific process is as follows: First, construct or obtain a validation dataset with correct annotations. This validation dataset contains several natural language input samples, each with a known correct expected output, used to determine whether the model's actual output is correct.

[0116] Secondly, a forward inference is performed once for each sample in the validation dataset using a large language model deployed on the front end, calculating the sequence-level uncertainty quantization score for each sample according to the method described above in this application. Simultaneously, the target output sequence generated by the model is compared with the correct label to obtain a binary classification result determining whether the model output is correct. Thus, a pair of data records is formed for each sample: the uncertainty quantization score and its corresponding correct / incorrect label.

[0117] Secondly, based on preset evaluation indicators, an optimal discrimination threshold, i.e., a preset safety threshold, is determined within the possible range of values ​​for the uncertainty quantification score. The preset evaluation indicators can be selected according to different preferences for risk and efficiency in different business scenarios. For example, a threshold that maximizes the proportion of correctly classified samples can be selected; or, in scenarios requiring high security, a threshold that guarantees a specific recall rate (such as identifying more than 95% of incorrect outputs) can be selected; or, in scenarios that balance efficiency, a threshold that achieves a preset balance between precision and recall can be selected.

[0118] Through the aforementioned offline verification process, the preset security threshold is given a clear statistical meaning, enabling it to reliably distinguish between high-confidence and low-confidence outputs in actual online services. This provides a reliable decision-making benchmark that aligns with the scenario's risk preferences for subsequent differentiated processing.

[0119] According to one embodiment of this application, the target output sequence includes a single token output sequence or a multi-token output sequence; In the scenario corresponding to the single token output sequence, the local soft margin score at the corresponding generation position is directly used as the sequence-level uncertainty quantization score.

[0120] Specifically, the length of the output sequence of a large language model varies fundamentally across different task paradigms. In scenarios such as free text generation, dialogue response, and summarization, the model typically needs to generate a target output sequence containing multiple semantic units per token; this is known as a multi-token output sequence. However, in discriminative or fill-in-the-blank tasks such as classification, intent recognition, sentiment analysis, and entity linking, the model output often consists of only a specific category label or entity identifier, and the corresponding target output sequence is composed of only a single token; this is known as a single-token output sequence.

[0121] In the generation logic of sequence-level uncertainty quantization scores, this application adopts adapted processing paths for the two output formats mentioned above.

[0122] For multi-token output sequences, as mentioned above, the overall credibility of the sequence is constrained by the weakest link in the autoregressive process. Therefore, it is necessary to perform sequence-level aggregation processing (such as taking the minimum value) on the local soft margin scores of each generation position to highlight the limiting effect of the generation position with the highest uncertainty on the overall output credibility.

[0123] The situation is different for single-token output sequences. In this scenario, the target output sequence contains only one generation position, and there are no issues of dependency propagation or risk accumulation between multiple positions. The credibility of the sequence is completely equivalent to the confidence level of this unique generation position. Therefore, continuing to use the aggregation logic applicable to multi-token scenarios is neither necessary nor practically meaningful.

[0124] Based on the above considerations, when determining that the target output sequence is a single-token output sequence, this application directly assigns the local soft margin score calculated for that generation position as the sequence-level uncertainty quantification score. In other words, in the single-token scenario, the position-level uncertainty measure and the sequence-level uncertainty measure are combined into one, without the need to introduce additional aggregation operators.

[0125] This approach ensures broad compatibility of the proposed solution across various task types. Whether the system is applied to generative dialogue scenarios requiring complete answers or classification scenarios requiring discrete labels, the uncertainty quantification module accurately outputs a unified score reflecting the credibility of the current inference result, providing standardized decision input for subsequent threshold comparisons and differentiated routing. Simultaneously, this design simplifies the system implementation logic, enabling the same uncertainty quantification framework to seamlessly adapt to multiple downstream tasks without requiring code branching or configuration switching for different task types.

[0126] According to one embodiment of this application, the downstream service processing includes a service interface call after intent recognition; When the sequence-level uncertainty quantization score is greater than or equal to the preset security threshold, the corresponding business interface is invoked according to the intent category represented by the target output sequence. When the sequence-level uncertainty quantization score is less than the preset security threshold, the business interface call is suspended.

[0127] Specifically, the downstream business processing includes calling business interfaces after intent recognition. In this scenario, the role of the large language model is to identify user intent from natural language input and represent the target output sequence as an intent category label. Based on the identified intent category, the system needs to call the corresponding business interface to perform subsequent operations, such as account query, order creation, information modification, or work order transfer.

[0128] This application directly links the comparison result of the sequence-level uncertainty quantization score and the preset security threshold to the execution decision of the business interface call, specifically in the following two scenarios: When the sequence-level uncertainty quantization score is greater than or equal to the preset security threshold, it indicates that the large language model has a high degree of confidence in the intent recognition result of the current natural language input, and the output intent category label is reliable. In this case, based on the intent category represented by the target output sequence, this application calls the business interface pre-associated with that intent category to execute the business process normally. This processing path ensures that the high-confidence recognition result can flow efficiently and drive business automation, giving full play to the advantages of the large language model in reducing the cost of manual intervention and improving response speed.

[0129] When the sequence-level uncertainty quantization score is less than the preset security threshold, it indicates that the large language model has significant uncertainty or hesitation in the intent recognition process, and the output intent category label has a high risk of error. In this case, if the business interface is called rashly based on this low-confidence label, it may lead to erroneous operations, such as misidentifying a user's intent to check their balance as an intent to transfer funds, or misidentifying a request to modify information as a request to cancel an account, thereby causing business losses, data errors, or security incidents.

[0130] To prevent the aforementioned risks, this application suspends the business interface calls when the sequence-level uncertainty quantification score is less than a preset security threshold. In other words, the system proactively blocks the automatic link from the intent recognition result to the business execution stage, preventing untrusted output from triggering actual business operations. Simultaneously, this application routes the corresponding request to a secure fallback channel, such as triggering model cascading scheduling for re-identification by a larger-scale model, transferring the request to human customer service for review, or guiding the user to clarify their intent through other interaction methods. By suspending business interface calls, this application establishes a security barrier for the business system under high uncertainty conditions, effectively reducing the automation risks caused by intent recognition errors and ensuring the security and reliability of business processing.

[0131] It should be noted that the aforementioned uncertainty-based business interface call control mechanism can be widely applied to various intelligent dialogue systems, voice assistants, intelligent customer service, and automated workflow platforms that use intent recognition as a pre-processing step. This mechanism transforms the uncertainty quantification score from an abstract evaluation indicator into a decision signal with direct business constraints, significantly improving the robustness and controllability of business systems based on large language models.

[0132] A second aspect of this application provides a reliable processing system for the output of a large language model based on local soft margins, comprising: The inference module receives natural language input and controls the large language model to perform a single forward inference to generate the target output sequence. The uncertainty quantization module is used to extract the original Logit score vector without Softmax normalization for each generation position during the generation process of the target output sequence, perform local candidate extraction processing on the original Logit score vector corresponding to each generation position, determine the target candidate and the suboptimal candidate set, determine the local soft margin score of the corresponding generation position based on the comprehensive competitive relationship between the target candidate and the suboptimal candidate set, and generate the sequence-level uncertainty quantization score based on the local soft margin score corresponding to each generation position. The threshold decision module is used to compare the sequence-level uncertainty quantization score with a preset safety threshold to obtain a release decision result or a backoff decision result; The routing execution module is used to output the target output sequence and trigger the corresponding downstream service processing when the release determination result is true, and to block the direct acceptance of the target output sequence when the fallback determination result is true, and to route the corresponding request to the safe fallback channel.

[0133] The uncertainty quantification module is configured as follows: The original Logit scores corresponding to each candidate token in the suboptimal candidate set are subjected to exponential transformation and summed, and then the summation result is subjected to logarithmic transformation to obtain the comprehensive competitive intensity corresponding to the suboptimal candidate set. The local soft margin score is determined based on the difference between the original Logit score of the target candidate and the overall competitiveness.

[0134] A second aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the embodiments of the first aspect above.

[0135] Figure 2 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 2 As shown, the electronic device may include a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the method in any of the embodiments of the first aspect described above. Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0136] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to perform the methods provided by the above methods.

[0137] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided by the above methods.

[0138] Example 2 This invention proposes a method based on Local Soft Margin (LS). M This invention relates to a method for quantifying uncertainty in the output of large language models and a trustworthy artificial intelligence system. The technical solution of this invention is described in detail below with specific method steps, system embodiments, and extended technical solutions.

[0139] 1. Overview of the Invention Principle To address the shortcomings of existing technologies, such as the high computational cost of black-box sampling methods, the structural loss of competitive information among multiple candidates due to excessive truncation in traditional white-box methods, and the distortion of competitive information among candidate tokens caused by Softmax forced normalization, this invention abandons the traditional approach of quantifying uncertainty in the probability space and directly performs feature extraction and computation in the original Logit score space output by the model.

[0140] The core technical concept of this invention is as follows: First, a Top-k truncation mechanism is used to filter out the top k candidate tokens and their corresponding Logit scores from the global vocabulary of a large language model (which typically contains tens of thousands to hundreds of thousands of tokens), thereby effectively isolating noise interference introduced by long-tail low-frequency tokens; second, a LogSumExp operation is applied to the second-best candidate set ranked 2nd to kth, and the Logit scores of multiple competitors are flexibly aggregated to obtain the comprehensive competitive strength of the second-best group; finally, the difference between the Logit score of the top-1 candidate token and the comprehensive competitive strength of the second-best group is calculated, i.e., LS. M Fraction Δ.

[0141] From a mathematical perspective, the LogSumExp function has the property of smoothing the maximum value: when there are multiple strong competitors with similar scores in the suboptimal set, its LogSumExp value will increase significantly due to the superposition effect of exponential summation, approaching or even exceeding the score of the Top-1 candidate, causing the Δ value to shrink sharply or turn into a negative value, indicating that the model has a high degree of uncertainty about the output; conversely, when the score of the Top-1 candidate token is significantly higher than that of all suboptimal candidates, the Δ value shows a large positive value, indicating that the model has a high degree of certainty about the output.

[0142] 2. Basic Technical Solution: Local Soft Margin Uncertainty Quantification Method like Figure 3 As shown, the uncertainty quantification method for large language model output based on local soft margins provided by this invention can be calculated in a single forward inference process of the model, without the need for additional sampling, training, or calibration steps. The specific calculation steps for each target token generated during the autoregressive decoding process are as follows: Step 1: Forward inference of the model and extraction of the original Logit score vector Receive natural language input prompts and feed them into the large language model. The model then performs autoregressive decoding to generate the first... When dealing with a target token, the raw Logit score vector generated by the model output layer (i.e., the linear layer at the head of the language model) without Softmax processing is intercepted and denoted as... Its dimension is equal to the size of the model vocabulary. Each element in this score vector The representation model represents the first word in the vocabulary. The original preference level of each token. In practice, if the model runs inference in low-precision floating-point format (such as bfloat16 or float16) to save GPU memory and computing power, then after extracting the score vector, it should be converted to single-precision floating-point (float32) format before subsequent calculations to ensure numerical accuracy.

[0143] Step 2: Local candidate set truncation and long-tail denoising Large language models typically have vocabulary sizes ranging from tens of thousands to hundreds of thousands of tokens. The vast majority of these tokens score extremely low at any given position, contributing nothing to uncertainty judgments and instead introducing noise. Therefore, for the score vector... All elements are sorted in descending order of value, and only the top-ranked elements are extracted. The Logit score corresponding to each candidate token. The Logit score corresponding to the token ranked 1st is denoted as... Ranked 2nd to 3rd The set of Logit scores corresponding to the remaining candidate tokens is denoted as .

[0144] Step 3: Calculation of Local Soft Margin Score Based on the local candidate set extracted in step two, the generated first... Local soft margins (LS) of a token M )Score The calculation formula is as follows: ; Among them, the right side of the formula The term refers to the LogSumExp function. This function is actually implemented using a numerically stable method, that is, first extracting... The maximum value in Then calculate: ; LS M Score The range of values ​​for is the real number field. The larger the value, the more significant the advantage of the Top-1 candidate token over the second-best candidate group, and the more certain the model's output for that position; The smaller the value (including negative values), the higher the uncertainty of the model's output at that position.

[0145] Step 4: Calculation of sequence-level uncertainty quantization score Depending on the application scenario, the present invention provides the following strategies: (a) Single-token output scenario: For tasks where the model output contains only a single token (e.g., output option identifiers A / B / C / D in a multiple-choice scenario), the LS calculated in step three... M Fraction That is, it can be directly used as the uncertainty quantification score of the output, without the need for aggregation.

[0146] (b) Multi-token output scenarios: For tasks where the model output contains multiple tokens (e.g., outputting a complete intent name string in an intent recognition scenario), calculate the LS of each token in the generated sequence. M Score , obtain the set ,in This represents the number of tokens in the output sequence. The tokens are aggregated using a min-pooling strategy. ; This strategy selects the sequence with the highest uncertainty ( The LS value of the token with the smallest value M The score serves as a quantification of the uncertainty of the entire sequence. Its technical basis lies in the fact that the overall reliability of a sequence output containing multiple tokens is constrained by the least reliable single token. High uncertainty in any one token can lead to semantic bias in the overall output. Other aggregation strategies are detailed in the extended technical solutions section.

[0147] 3. Preferred technical solution: A trustworthy artificial intelligence system based on local soft margins Based on the aforementioned method for quantifying local soft margin uncertainty, this invention further provides a trustworthy artificial intelligence system based on uncertainty perception. This system utilizes the local soft margin (LS) uncertainty quantification method. M )Score As the basis for dynamic routing decisions, it automatically determines whether to accept the output result or trigger a safety rollback based on the degree of determinism of the model output, and can flexibly adapt to the reliability and cost requirements of different business scenarios by adjusting the threshold.

[0148] 3.1 System Architecture like Figure 4 As shown, the trusted artificial intelligence system includes the following functional modules: (1) Inference module: Deploy a small-parameter edge-side large language model (e.g., a lightweight model with 4 billion parameters), receive natural language requests from users, perform a single forward inference, and synchronously output the results and the corresponding original Logit score vector. .

[0149] (2) Uncertainty Quantification Module: Following the methods described in steps one to four of the basic technical solution above, the Top-k candidate set is extracted from the original Logit score vector output by the inference module, and the LS is calculated. M Score The computational complexity of this module is extremely low, involving only one Top-k sorting operation and one LogSumExp operation, which is negligible compared to the computational cost of the model's forward inference itself.

[0150] (3) Threshold Decision Module: The LS output from the uncertainty quantification module is used to make the decision. M Score With preset safety threshold The comparison is performed, and different routing strategies are executed based on the comparison results.

[0151] (4) Routing Execution Module: Based on the determination result of the threshold decision module, execute one of the following two paths: Release path: when If the reliability of the model output is deemed to meet the requirements, the output result is directly accepted and the downstream business interface is called to perform subsequent operations.

[0152] rollback path: when If the model output is deemed to have high uncertainty, the output result is intercepted, and the request is forwarded to the safe rollback channel.

[0153] 3.2 Implementation of the safe rollback channel The secure rollback channel supports multiple implementation methods and can be used individually or in combination according to specific business needs, for example: (A) Model Cascade Scheduling: The request is automatically forwarded to a high-parameter, large-scale language model (e.g., a model with hundreds of billions of parameters) deployed in the cloud for secondary inference. Through a cascade architecture of "lightweight model pre-screening and advanced model intervention on demand", the overall system's high accuracy is maintained while controlling the proportion of high-cost model calls, thereby reducing the overall system operating cost.

[0154] (B) Manual intervention routing: The request and related context information are routed to the manual processing channel, where a final result is given after manual review, ensuring the reliability of the system output.

[0155] In practical deployments, other methods can also be used, such as returning security alerts to the user and requesting supplementary information (e.g., further clarification of intent or additional context) to reduce uncertainty in subsequent reasoning. This invention does not limit the specific implementation of the secure fallback channel.

[0156] 3.3 Threshold Determination Method Safety threshold The determination was made using offline validation: before system deployment, a set of correctly labeled validation datasets was used to calculate the LS of each sample. M Score And record whether the model output is correct. Based on business needs, weigh accuracy against rollback ratio, and select the option that optimizes the target evaluation metric. Value as Under different levels of business risk tolerance, The strategy for setting the value differs: in scenarios requiring high accuracy (such as in the medical or financial fields), a higher value should be set. The value is set so that only highly deterministic outputs are directly accepted, while all other requests enter a safe fallback path, thereby minimizing the risk of erroneous outputs. In scenarios where tolerance for individual errors is higher and system processing efficiency is more important, the value can be appropriately reduced. This reduces the rollback rate, allowing more requests to be processed automatically by the system.

[0157] 3.4 Application Scenario Example: Intent Recognition Gateway Taking the intent recognition scenario of an enterprise-level intelligent assistant as an example, the specific deployment method of the above system is illustrated. The system receives natural language commands from end users (such as "Help me check how many days of annual leave I have left"), calls a lightweight large language model deployed on the edge to perform intent classification (such as determining it as the intent "annual leave balance query"), and simultaneously calculates the LS of the output. M The score is Δ. If Δ ≥ τ, the system directly calls the HR system API to return the remaining annual leave balance; if Δ < τ, the system does not execute the API call, but instead forwards the request to a cloud-based advanced model for secondary judgment and other safety fallback channels.

[0158] 3.5 Verification Implementation Example: Verification of Technical Effect In a specific verification embodiment, the Qwen3-4B-Instruct model (with 4 billion parameters) is used as the base model for the inference module. A greedy decoding strategy is employed, and the Top-k parameters are set to... The LS of this invention was tested on two natural language understanding datasets. M The discriminative power of our method is compared with existing white-box uncertainty quantification methods. The evaluation metric is AUROC (Area Under Receiver Operating Characteristic), which measures the ability of each method to rank reliable outputs from hallucinatory outputs.

[0159] like Figure 5 As shown, the present invention LS M The following are the results comparing the discriminative power of the method with existing white-box methods on two datasets: On the MCQ dataset (25,318 single-choice questions), LS... M The AUROC reached 0.7823, an improvement of 1.15 percentage points compared to the second-best negative entropy method (0.7708). On the SNIPS intent recognition dataset (1,400 test samples covering 7 intent categories), LS... M The AUROC reached 0.8717, which is 4.57 percentage points higher than the second-best negative entropy method (0.8260), showing a particularly significant advantage.

[0160] Furthermore, the LS method of this invention is applied to the SNIPS dataset.M The method (single-inference) is compared with black-box uncertainty quantification methods that require multiple sampling. The black-box method employs a random sampling strategy with a temperature parameter of 1.2 and a Top-p value of 0.95, generating 5 independent responses for each sample. Figure 6 As shown, LS M The AUROC of 0.8717 is achieved with only a single forward inference pass, significantly outperforming the Monte Carlo sequence probability method (AUROC 0.8320), which requires five times the inference cost, and far exceeding the semantic entropy method (AUROC 0.5558). These results verify the technical advantages of this invention, which balances computational efficiency and discriminative power, demonstrating that LS... M The method is suitable as the core algorithm for the uncertainty quantification module in the aforementioned trusted artificial intelligence system.

[0161] 4. Extended technical solutions and implementation variations The technical solution of the present invention is not limited to the above basic implementation method. Several extended variations are listed below: 4.1 Variations of the Top-k truncation mechanism In the above basic scheme, For fixed hyperparameters (e.g.) In the extended implementation, This can be replaced by a dynamically determined candidate set size. For example, drawing inspiration from the Top-p concept in Nucleus Sampling, a Softmax operation can be applied to the original Logit score vector to obtain a probability distribution, which can then be accumulated from highest to lowest probability until the cumulative probability reaches a preset threshold. (like ), to accumulate probability to The Logit scores corresponding to all previous tokens are used as a candidate set, and then Local Soft Margin (LS) is applied. M The score is calculated. This variant allows the candidate set size to automatically adjust based on the concentration of the model output distribution.

[0162] 4.2 Variants of the LogSumExp aggregate function The LogSumExp function can introduce a temperature coefficient. Scaling is performed to scale the LS in step three of the basic technical solution. M The scoring formula is generalized as follows: ; when It degenerates into a basic scheme; when As it approaches 0, LogSumExp approaches the HardMaximum, LS M The score Δ degenerates into the difference between the Top-1 and Top-2 scores; when As the value increases, candidate tokens with lower scores in the suboptimal candidate set receive higher weights in the LogSumExp aggregation. Temperature coefficient. It can be used as an adjustable hyperparameter to optimize according to the specific task characteristics. In addition, LogSumExp can also be replaced by other aggregation operations that can measure the overall competitive strength of the suboptimal candidate set, such as the arithmetic mean or weighted mean of the scores of the suboptimal candidate set. The technical essence of these operations is to separate and compare the local candidate set in the Logit score space.

[0163] 4.3 Variations of Sequence-Level Aggregation Strategies In scenarios with multiple token outputs, besides the minimum pooling in the basic approach, variations such as average pooling and weighted pooling can also be used. For example, in some application scenarios where sensitivity to single-point extreme values ​​is low, an average pooling strategy can be adopted. This strategy reflects the average uncertainty level of the entire sequence. The weighted pooling aggregation strategy applies the LLS (Laser Scale) of each token in the sequence. M The score Δ is assigned different weights according to its position in the sequence to highlight the impact of key tokens on the overall uncertainty of the sequence.

[0164] 4.4 Combination with post-processing calibration methods The LS output by this invention M The score is the difference in the Logit score space, which does not have probabilistic meaning. In applications where it is necessary to convert uncertainty quantification scores into confidence scores with probabilistic meaning, LS can be used. M A post-processing calibration layer is then overlaid on the score. For example, using Platt scaling or isotonic regression methods, based on a set of labeled calibration datasets, the LS... M The score is mapped to the calibrated probability value. At this point, LS... M The method, acting as a feature extraction layer, provides raw scores with good ranking capabilities, while the calibration method, acting as a post-processing layer, transforms the scores into probabilities. The two layers are independent of each other. LS M The method itself can perform the sorting and filtering functions of uncertainty quantification without relying on the calibration step.

[0165] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0166] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for processing the output of a large language model based on local soft margins, characterized in that, include: Receive natural language input and control a large language model to perform a single forward inference based on the natural language input to generate a target output sequence; During the generation of the target output sequence, the original Logit score vector, which is not Softmax normalized, is extracted from the output of the large language model at each generation position. Local candidate extraction processing is performed on the original Logit score vector corresponding to each generation position to select a preset number of candidate tokens with the highest scores from all candidates corresponding to the vocabulary, forming a local candidate set for the corresponding generation position, and determining the target candidate ranked first and the set of second-best candidates other than the target candidate in the local candidate set. Based on the original Logit score of the target candidate and the original Logit score of each candidate token in the suboptimal candidate set, the comprehensive competitive relationship between the target candidate and the suboptimal candidate set is flexibly aggregated to obtain the local soft margin score of the corresponding generation position, so that the local soft margin score simultaneously represents the leading degree of the target candidate and the competitive intensity formed by multiple suboptimal candidates against the target candidate. When the target output sequence contains multiple generation positions, the local soft margin scores corresponding to each generation position are subjected to sequence-level aggregation processing to obtain the sequence-level uncertainty quantification score corresponding to the target output sequence. The sequence-level aggregation processing is used to highlight the limiting effect of the generation position with the highest uncertainty in the target output sequence on the overall output credibility. When the target output sequence contains only one generation position, the local soft margin score corresponding to that generation position is used as the sequence-level uncertainty quantization score. The sequence-level uncertainty quantization score is compared with a preset security threshold, and differential processing is performed on the target output sequence based on the comparison result. Specifically, when the sequence-level uncertainty quantization score is greater than or equal to the preset security threshold, the target output sequence is output and downstream business processing corresponding to the target output sequence is triggered; when the sequence-level uncertainty quantization score is less than the preset security threshold, direct acceptance of the target output sequence is blocked, and the request corresponding to the natural language input is routed to a secure fallback channel. The construction of the local candidate set, the determination of the local soft margin score, the generation of the sequence-level uncertainty quantization score, and the differential processing based on the sequence-level uncertainty quantization score are all related processing processes that are continuously executed for the same forward inference result. The process of obtaining the local soft margin score is always completed within the local candidate set based on the original Logit score vector. No probability normalization processing is performed on the local candidate set, and no multiple sampling and generation are performed for the same natural language input.

2. The method according to claim 1, characterized in that, The local candidate extraction process for the original Logit score vector corresponding to each generated position includes: The scores corresponding to each candidate token in the original Logit score vector are sorted by size, and the top k candidate tokens are selected as the local candidate set, where k is a preset integer greater than 2.

3. The method according to claim 1, characterized in that, The target candidate is the candidate token ranked first in the local candidate set, and the suboptimal candidate set includes the remaining candidate tokens in the local candidate set other than the target candidate.

4. The method according to claim 1, characterized in that, The flexible aggregation processing of the comprehensive competitive relationship between the target candidate and the set of suboptimal candidates includes: Perform an exponential transformation on the original Logit scores corresponding to each candidate token in the suboptimal candidate set and sum them, then perform a logarithmic transformation on the summation result to obtain the comprehensive competitive intensity corresponding to the suboptimal candidate set; The local soft margin score is determined based on the difference between the original Logit score of the target candidate and the overall competitiveness.

5. The method according to claim 1, characterized in that, The sequence-level aggregation of the local soft margin scores corresponding to each generated position includes: The minimum value among the local soft margin scores corresponding to each generation position is determined, and the minimum value is used as the sequence-level uncertainty quantization score.

6. The method according to claim 1, characterized in that, After extracting the original Logit score vector, the following is also included: When the large language model performs inference in low-precision floating-point format, the original Logit score vector is converted into single-precision floating-point format, and then the local candidate extraction process and the local soft margin score determination process are performed.

7. The method according to claim 1, characterized in that, The safe rollback channel includes a model cascade scheduling channel, which forwards the corresponding request to another large language model with a larger parameter scale than the large language model for secondary inference when the sequence-level uncertainty quantization score is less than the preset safety threshold. The preset security threshold is determined through offline verification, specifically including: Based on the verification dataset with correct annotations, the sequence-level uncertainty quantification score corresponding to each sample is calculated, and the correctness of the model output is determined in combination with the preset evaluation index to determine the preset safety threshold.

8. The method according to claim 1, characterized in that, The target output sequence includes a single token output sequence or a multi-token output sequence; In the scenario corresponding to the single token output sequence, the local soft margin score at the corresponding generation position is directly used as the sequence-level uncertainty quantization score.

9. The method according to claim 1, characterized in that, The downstream business processing includes business interface calls after intent recognition; When the sequence-level uncertainty quantization score is greater than or equal to the preset security threshold, the corresponding business interface is invoked according to the intent category represented by the target output sequence. When the sequence-level uncertainty quantization score is less than the preset security threshold, the business interface call is suspended.

10. A large language model output processing system based on local soft margins, characterized in that, include: The inference module receives natural language input and controls the large language model to perform a single forward inference to generate the target output sequence. The uncertainty quantization module is used to extract the original Logit score vector without Softmax normalization for each generation position during the generation process of the target output sequence, perform local candidate extraction processing on the original Logit score vector corresponding to each generation position, determine the target candidate and the suboptimal candidate set, determine the local soft margin score of the corresponding generation position based on the comprehensive competitive relationship between the target candidate and the suboptimal candidate set, and generate the sequence-level uncertainty quantization score based on the local soft margin score corresponding to each generation position. The threshold decision module is used to compare the sequence-level uncertainty quantization score with a preset safety threshold to obtain a release decision result or a backoff decision result; The routing execution module is used to output the target output sequence and trigger the corresponding downstream service processing when the release determination result is true, and to block the direct acceptance of the target output sequence when the fallback determination result is true, and to route the corresponding request to the safe fallback channel. The uncertainty quantification module is configured as follows: Perform an exponential transformation on the original Logit scores corresponding to each candidate token in the suboptimal candidate set and sum them, then perform a logarithmic transformation on the summation result to obtain the comprehensive competitive intensity corresponding to the suboptimal candidate set; The local soft margin score is determined based on the difference between the original Logit score of the target candidate and the overall competitiveness.

Images