Large model uncertainty estimation method and system based on semantic preserving disturbance
Through the semantic persistence method, the uncertainty of the output of large language models is quantified, and the problem of difficult to distinguish between cognitive and accidental uncertainty in the prior art is solved, and more efficient uncertainty estimation is achieved.
Patent Information
- Application Number
- CN202510528919.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-25
- Publication Date
- 2025-08-01
- Estimated Expiration
- 2045-04-25
AI Technical Summary
The existing uncertainty estimation method is difficult to accurately distinguish between cognitive uncertainty and accidental uncertainty in the free generation task of large language models, and requires the reconstruction of huge output space, resulting in inefficiency.
Through the method of semantic preservation perturbation, multiple texts with the same semantics but different expressions are generated by rewrite models or random character deletion, calculate the predicted distribution changes of the output token, and quantify uncertainty.
Effectively distinguishing cognitive uncertainty from accidental uncertainty improves the accuracy and efficiency of uncertainty estimation and avoids the need for reconstruction of huge output space.
Smart Images
Figure CN120408116A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of natural language processing, and specifically relates to a method and system for estimating the uncertainty of large models based on semantically-preserving perturbations. Background Art
[0002] In recent years, large language models (LLMs) have developed rapidly and demonstrated amazing capabilities in traditional natural language processing tasks. However, they still face the challenge of generating incorrect facts. Even the most powerful large models often generate untrue content, a phenomenon commonly known as 'hallucination', which significantly reduces the reliability of large models and limits their application scope.
[0003] Uncertainty Quantification (UQ) is considered a promising direction for improving the reliability of large models. UQ uses technical means to measure the uncertainty score of a large model regarding the correctness of its own output. The estimated uncertainty scores can be applied to multiple fields, such as error detection, active learning, and selective generation, etc., so as to improve the reliability of the model. Uncertainty is generally divided into two categories: aleatoric uncertainty (also known as data uncertainty) and epistemic uncertainty (also known as model uncertainty). Aleatoric uncertainty stems from the inherent and irreducible randomness in the data (for example, there may naturally be multiple reasonable answers to a certain question). Epistemic uncertainty, on the other hand, results from insufficient understanding of the underlying data generation process, which may be attributed to factors such as insufficient training or distribution shift between the training set and the test set. Epistemic uncertainty is generally considered a more reliable indicator of model credibility.
[0004] Although the effectiveness of uncertainty estimation in classification and regression tasks has been verified, applying uncertainty estimation in large model free generation tasks is not straightforward. This challenge stems from the inherent characteristics of natural language, such as the complexity of the output space and the natural entanglement of epistemic uncertainty and aleatoric uncertainty. Existing research on uncertainty estimation in free generation tasks mainly relies on measuring semantic changes within the output space. These methods need to reconstruct an extremely large output space through sampling and have difficulty distinguishing epistemic uncertainty from aleatoric uncertainty. Summary of the Invention
[0005] The present invention provides a method for estimating the uncertainty of large models based on semantically-preserving perturbations to solve the problem of inaccurate uncertainty estimation in large language model free generation tasks due to the inherent characteristics of natural language.
[0006] The present invention provides a system for estimating the uncertainty of large models based on semantically-preserving perturbations to implement the method for estimating the uncertainty of large models based on semantically-preserving perturbations.
[0007] The present invention is achieved through the following technical solutions:
[0008] A method for estimating the uncertainty of a large model based on semantic-preserving perturbation, the estimation method comprising the following steps:
[0009] Step 1: Perform semantic-preserving intervention by using the large model to rewrite or input through SOC;
[0010] Step 2: Based on the semantic-preserving intervention in Step 1, calculate the uncertainty score;
[0011] Step 3: Based on the score calculated in Step 2, achieve the uncertainty estimation of the large model based on semantic-preserving perturbation.
[0012] Further, Step 1 is specifically as follows: The input part that needs to be input into the large model is first input into another independent rewriting model;
[0013] If the input part is in Chinese, the rewriting model is used to generate multiple texts with the same semantics but different expressions for the input part;
[0014] If the input part is in English, words are randomly selected from the input text, and one or more English characters are randomly deleted from each selected word after SOC, then the English input part is obtained.
[0015] Further, Step 2 is specifically as follows: Let the original input of the model be x, and the output be y = {y1, y2, …, y T},
[0016] where y1, y2, …, y T are the tokens generated by the model;
[0017] Write the prediction distribution of the model generating the t-th token as p(y|y <t , x),
[0018] where y <t = {y1, y2, …, y t-1} is the first t - 1 tokens generated by the model;
[0019] Write the text with the same semantics as the input x as
[0020] Further, given the input x, use the semantic-preserving intervention method to generate L texts with the same semantics as x but different expressions
[0021] Obtain the output y * of the model based on the original input x and the prediction distribution of each token
[0022]
[0023] Among them
[0024] Concatenate y * with each perturbed input to obtain the predicted distribution of each output token after perturbation by inputting into the model
[0025] Use the average change in the predicted distribution of tokens before and after perturbation as the uncertainty estimation score.
[0026] Furthermore, the specific method of using the average change in the predicted distribution of tokens before and after perturbation as the uncertainty estimation score is
[0027]
[0028] where D H is a metric function for the difference between two probability distributions, and α t is the information weight
[0029] A large model uncertainty estimation system based on semantically-preserving perturbation. The system uses the large model uncertainty estimation method based on semantically-preserving perturbation as described above. The system includes:
[0030] Semantic intervention module: Perform semantically-preserving intervention by using the large model to rewrite or SOC input;
[0031] Uncertainty scoring calculation module: Calculate the uncertainty score based on the semantically-preserving intervention;
[0032] Large model uncertainty estimation implementation module: Implement the large model uncertainty estimation based on semantically-preserving perturbation based on the calculated score.
[0033] Furthermore, the working principle of the uncertainty scoring calculation module is as follows. Let the original input of the model be x, and the output be y = {y1, y2,..., y T},
[0034] where y1, y2,..., y T are the tokens generated by the model;
[0035] Write the predicted distribution of the t-th token generated by the model as p(y|y <t , x),
[0036] where y <t = {y1, y2,..., y t-1} are the first t - 1 tokens generated by the model;
[0037] Write the text with the same semantics as input x as
[0038] Furthermore, given an input x, a semantics-preserving intervention method is used to generate L texts with the same semantics as x but different expressions.
[0039] Get the model's output y based on the original input x * And the predicted distribution of each token
[0040]
[0041] in
[0042] y * With each perturbed input Splicing, the input model gets the predicted distribution of each output token after perturbation
[0043] The change in token prediction distribution before and after the average perturbation is used as the uncertainty estimation score;
[0044] The change in token prediction distribution before and after the average disturbance is used as the uncertainty estimation score, specifically:
[0045]
[0046] Among them, D H is the measurement function of the difference between two probability distributions, α t is the information weight,
[0047] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described above is implemented.
[0048] .A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method as described above is implemented.
[0049] The beneficial effects of the present invention are:
[0050] The present invention actually estimates the expectation of the difference in output distribution before and after the disturbance, which can effectively distinguish epistemic uncertainty from aleatory uncertainty.
[0051] The present invention is different from previous methods in that it avoids the need to reconstruct a large and complex output space, thereby achieving higher stability and efficiency.
[0052] The present invention quantifies the average change in the prediction distribution of the same output token before and after semantic-preserving intervention. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 is a schematic structural diagram of the present invention.
[0054] Figure 2 is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0055] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.
[0056] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.
[0057] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.
[0058] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings of the present application specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.
[0059] Many specific details are set forth in the following description in order to provide a thorough understanding of the present application, but the present application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.
[0060] Embodiment 1
[0061] This embodiment provides a method for estimating the uncertainty of a large model based on semantic-preserving perturbations, as Figure 1 - Figure 2As shown, the estimation method includes the following steps:
[0062] Step 1: Perform semantic-preserving intervention by using a large model for rewriting or SOC input.
[0063] Step 2: Calculate the uncertainty score based on the semantic-preserving intervention in Step 1.
[0064] Step 3: Based on the score calculated in Step 2, achieve the uncertainty estimation of the large model based on semantic-preserving perturbation.
[0065] Further, Step 1 is specifically as follows: First, input the part to be input into the large model into another independent rewriting model.
[0066] If the input part is in Chinese, the rewriting model is used to generate multiple texts with the same semantics but different expressions for the input part.
[0067] If the input part is in English, randomly select words from the input text, and randomly delete one or several English characters SOC for each selected word, then the English input part is obtained.
[0068] Experimental proofs based on semantic similarity show that both methods can effectively disrupt the surface structure of the text while maintaining semantic similarity.
[0069] Further, Step 2 is specifically as follows:
[0070] Let the original input of the model be x, and the output be y = {y1, y2, …, y T}, where y1, y2, …, y T are the tokens generated by the model.
[0071] Write the prediction distribution of the model for generating the t-th token as p(y|y <t , x),
[0072] where y <t = {y1, y2, …, y t-1} is the first t - 1 tokens generated by the model.
[0073] Write the text with the same semantics as the input x as
[0074] Further, given the input x, use the semantic-preserving intervention method to generate L texts with the same semantics as x but different expressions
[0075] Obtain the output y * of the model based on the original input x and the prediction distribution of each token
[0076]
[0077] Among them
[0078] Combine y * with each perturbed input and splice them. Input the model to obtain the predicted distribution of each output token after perturbation
[0079] Take the average change in the predicted distribution of tokens before and after perturbation as the uncertainty estimation score.
[0080] Furthermore, taking the average change in the predicted distribution of tokens before and after perturbation as the uncertainty estimation score is specifically as follows
[0081]
[0082] Among them, D H is a metric function for the difference between two probability distributions. Here, the Hellinger distance is adopted. α t is the information weight, which is used to penalize tokens with low information density and low importance. Here, the entropy of the predicted distribution function is taken as the weight, that is
[0083] A more specific embodiment is as follows
[0084] Experiments were conducted on multiple question-answering datasets, including SciQ, TriviaQA, CoQA, AmbigQA, and TruthfulQA. The uncertainty scores were used to predict whether the model's responses were correct, that is, to determine whether the model's output could be trusted.
[0085] The higher the prediction accuracy, the more the score can truly represent the confidence level of the model in its output. The common evaluation metric AUROC for classification tasks was used as the evaluation metric. Experiments were conducted on two current mainstream large models of different sizes, and the proposed method was compared with four baseline methods, namely: Length-Normalized Prediction Entropy (LN-PE), INSIDE, Semantic Entropy, and SAR. LN-PE directly uses Monte Carlo estimation to calculate the output space entropy based on the length-normalized sentence log probability. INSIDE utilizes the inherent variation in the semantic embeddings of the sampled outputs to quantify uncertainty. Semantic Entropy considers semantic equivalence and estimates the output space entropy after clustering semantically equivalent outputs. SAR is the current state-of-the-art method, which introduces importance weights and shifts the attention to more relevant words and sentences, thereby improving the uncertainty scoring.
[0086] The final experimental results are shown in Figure 1
[0087]
[0088]
[0089] Through extensive experiments on five datasets, it is proven that the algorithm's performance is consistently superior to existing state-of-the-art methods, demonstrating the effectiveness of the algorithm.
[0090] Embodiment 2
[0091] This embodiment provides a large model uncertainty estimation system based on semantic-preserving perturbation. The system uses the large model uncertainty estimation method based on semantic-preserving perturbation described in Embodiment 1, as Figure 1 shown. The system includes:
[0092] Semantic intervention module: Perform semantic-preserving intervention by means of large model rewriting or SOC input;
[0093] Uncertainty score calculation module: Calculate the uncertainty score based on the semantic-preserving intervention;
[0094] Large model uncertainty estimation implementation module: Implement the large model uncertainty estimation based on semantic-preserving perturbation based on the calculated score.
[0095] Furthermore, the working principle of the uncertainty score calculation module is as follows. Let the original input of the model be x, and the output be y = {y1, y2,..., y T},
[0096] where y1, y2,..., y T are the tokens generated by the model;
[0097] Write the predicted distribution of the model generating the t-th token as p(y|y <t , x),
[0098] where y <t = {y1, y2,..., y t-1} is the first t - 1 tokens generated by the model;
[0099] Write the text with the same semantics as the input x as
[0100] Furthermore, given the input x, use the semantic-preserving intervention method to generate L texts with the same semantics as x but different expressions
[0101] Obtain the output y * of the model based on the original input x and the predicted distribution of each token
[0102]
[0103] wherein
[0104] concatenate y * with each perturbed input to obtain the predicted distribution of each output token after perturbation when input into the model
[0105] Use the average change in the predicted distribution of tokens before and after perturbation as the uncertainty estimation score;
[0106] The specific method of using the average change in the predicted distribution of tokens before and after perturbation as the uncertainty estimation score is that
[0107]
[0108] where D H is a metric function for the difference between two probability distributions, and α t is the information weight
[0109] Embodiment 3
[0110] The embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the above memory and executable on the above processor. Among them, the memory is used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory and the processor are connected by a bus. Specifically, when the processor runs the above computer program stored in the memory, any step in Embodiment 1 above is implemented.
[0111] It should be understood that in the embodiment of the present invention, the so-called processor may be a central processing unit (CPU), and this processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.
[0112] The memory may include a read-only memory, a flash memory, and a random access memory, and provide instructions and data to the processor. Part or all of the memory may also include a non-volatile random access memory.
[0113] As can be seen from the above, the electronic device provided by the embodiment of the present invention can implement the large model uncertainty estimation method based on semantic-preserving perturbation as described in Embodiment 1 by running a computer program. The proposed algorithm starts from a causal perspective and establishes a connection between the uncertainty of the large model and the strength of the causal path controlling its reasoning process. The intuition is that the deeper the model's understanding of the underlying causal mechanism, the lower the uncertainty of the generated response. Based on this premise, a new method for large model uncertainty quantification (UQ) is proposed, which quantifies the uncertainty by measuring the invariance of the output after applying semantic-preserving interventions to the model inputs (prompts). Specifically, the average change in the prediction distribution of the same output token before and after the semantic-preserving intervention is quantified.
[0114] It should be understood that if the above integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes of the above method embodiments of the present invention can also be completed by a computer program instructing relevant hardware. The above computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the above computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The above computer-readable medium may include: any entity or device capable of carrying the above computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the above computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.
[0115] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of mutual distinction and do not limit the protection scope of the present invention. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.
[0117] It should be noted that the methods and their detailed examples provided in the above embodiments can be combined with the devices and equipment provided in the embodiments, and reference can be made to each other and will not be elaborated herein.
[0118] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.
[0119] In the embodiments provided by the present invention, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / equipment embodiments described above are only illustrative. For example, the above-mentioned division of modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
[0120] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method for estimating the uncertainty of a large model based on semantically-preserving perturbations, characterized in that The estimation method includes the following steps: Step 1: Perform semantic-preserving intervention by means of large model rewriting or SOC input; Step 2: Calculate the uncertainty score based on the semantic-preserving intervention in Step 1; Step 3: Realize the uncertainty estimation of the large model based on semantic-preserving perturbation based on the score calculated in Step 2.
2. The method according to claim 1, wherein Specifically, for Step 1, the input part to be input into the large model is first input into another independent rewriting model; If the input part is in Chinese, the rewriting model is used to generate multiple texts with the same semantics but different expressions for the input part; If the input part is in English, words are randomly selected from the input text, and one or several English characters are randomly deleted from each selected word after SOC, then the input part in English is obtained.
3. The method according to claim 1, wherein Specifically, in step 2, let the original input of the model be x, and the output be y = {y1, y2, …, y T} Among them, y1, y2, …, y T are tokens generated by the model; Write the prediction distribution of the model generating the t-th token as p(y|y<t,x), Among them, y<t={y1,y2,…,y t-1} are the first t-1 tokens generated by the model; Write the text with the same semantics as the input x as 4. The method according to claim 3, wherein Given the input x, use the semantic-preserving intervention method to generate L texts that have the same semantics as x but different expressions Obtain the output y of the model based on the original input x * and the predicted distribution of each token Among them Concatenate y * with each perturbed input to obtain the predicted distribution of each output token after perturbation when input into the model Take the average change in the token prediction distribution before and after perturbation as the uncertainty estimation score.
5. The method according to claim 4, characterized in that The specific method of taking the average change in the token prediction distribution before and after perturbation as the uncertainty estimation score is as follows: where D H is a metric function for the difference between two probability distributions, and α t is the information weight, 6. A large model uncertainty estimation system based on semantic-preserving perturbation, characterized in that, The system uses the large model uncertainty estimation method based on semantic-preserving perturbation as described in any one of claims 1-5. The system includes: Semantic intervention module: Perform semantic-preserving intervention by means of large model rewriting input; Uncertainty score calculation module: Calculate the uncertainty score based on the semantic-preserving intervention; Large model uncertainty estimation implementation module: Realize the uncertainty estimation of the large model based on semantic-preserving perturbation based on the calculated score.
7. The system according to claim 6, wherein The working principle of the uncertainty score calculation module is as follows. Let the original input of the model be x, and the output be y = {y1, y2, …, y T}, where y1, y2, …, y T are tokens generated by the model; Write the prediction distribution of the model generating the t-th token as p(y|y<t,x), Among them, y<t={y1,y2,…,y t-1} are the first t - 1 tokens generated by the model; Write the text with the same semantics as the input x as 8. The system according to claim 7, wherein Given the input x, use the semantic-preserving intervention method to generate L texts that have the same semantics as x but different expressions Obtain the output y of the model based on the original input x * and the predicted distribution of each token Among them Concatenate y * with each perturbed input to obtain the predicted distribution of each output token after perturbation when input into the model Take the average change in the token prediction distribution before and after perturbation as the uncertainty estimation score; The specific method of taking the average change in the token prediction distribution before and after perturbation as the uncertainty estimation score is as follows: Among them, D H is a measure function of the difference between two probability distributions, and α t is the information weight, 9. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method as described in any one of claims 1-5 is realized.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method as described in any one of claims 1-5 is realized.