Semantic-preserving perturbation-based large model uncertainty estimation method and system

Abstract

The present application belongs to the technical field of natural language processing, and particularly relates to a large model uncertainty estimation method and system based on semantic preservation disturbance. Step 1: semantic preservation intervention is performed in the manner of large model rewriting or SOC input; Step 2: based on the semantic preservation intervention of step 1, uncertainty score calculation is performed; Step 3: based on the score calculated in step 2, large model uncertainty estimation based on semantic preservation disturbance is realized. The present application is used to solve the problem that due to the inherent characteristics of natural language, the uncertainty estimation in the free generation task of a large language model is inaccurate.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to a method and system for estimating the uncertainty of large models based on semantically preserved perturbations. Background Technology

[0002] In recent years, large language models (LLMs) have developed rapidly and demonstrated amazing capabilities in traditional natural language processing tasks. However, they still face the challenge of generating false facts. Even the most powerful large models often generate unrealistic content, a phenomenon often referred to as 'illusion'. This significantly reduces the reliability of large models and limits their application scope.

[0003] Uncertainty Quantification (UQ) is considered a promising direction for improving the reliability of large models. Uncertainty estimation uses techniques to measure the uncertainty score of a large model regarding the correctness of its own output. The estimated uncertainty score can be applied to multiple fields, such as error detection, active learning, and selective generation, thereby improving model reliability. Uncertainty is generally divided into two categories: random uncertainty (also known as data uncertainty) and cognitive uncertainty (also known as model uncertainty). Random uncertainty stems from the inherent, unreducible randomness in the data (e.g., multiple reasonable answers may naturally exist for a given question). Cognitive uncertainty, on the other hand, stems from insufficient understanding of the underlying generation process of the real data, which may be attributed to insufficient training or distributional shifts between the training and test sets. Cognitive uncertainty is generally considered a more reliable indicator of model credibility.

[0004] While the effectiveness of uncertainty estimation in classification and regression tasks has been validated, applying it to large-scale, freely generated tasks is not straightforward. This challenge stems from the inherent characteristics of natural language, such as the complexity of the output space and the natural interweaving of cognitive and accidental uncertainties. Existing research on uncertainty estimation in freely generated tasks primarily relies on measuring semantic variations within the output space. These methods require reconstructing extremely large output spaces through sampling and struggle to distinguish between cognitive and accidental uncertainties. Summary of the Invention

[0005] This invention provides a method for estimating the uncertainty of large models based on semantically preserved perturbations, in order to solve the problem of inaccurate uncertainty estimation in the free generation task of large language models due to the inherent characteristics of natural language.

[0006] This invention provides a large model uncertainty estimation system based on semantically preserved perturbations, which is used to implement a large model uncertainty estimation method based on semantically preserved perturbations.

[0007] This invention is achieved through the following technical solution:

[0008] A method for estimating the uncertainty of a large model based on semantically preserved perturbations, the estimation method comprising the following steps:

[0009] Step 1: Use large model rewriting or SOC input to perform semantic preservation intervention;

[0010] Step 2: Based on the semantic preservation intervention in Step 1, calculate the uncertainty score;

[0011] Step 3: Based on the score calculated in Step 2, perform large model uncertainty estimation based on semantically preserved perturbation.

[0012] Furthermore, step 1 specifically involves inputting the input portion that needs to be input into the large model into another independent rewritten model;

[0013] If the input is in Chinese, the model is rewritten to generate multiple texts with the same semantics but different expressions.

[0014] If the input is in English, words are randomly selected from the input text. One or more English characters are randomly deleted from each selected word after SOC, and then the English input is obtained.

[0015] Furthermore, step 2 specifically involves assuming the original input of the model is x, and the output is y = {y1, y2, ..., y...}. T},

[0016] Where y1, y2, ..., y T The token generated for the model;

[0017] Let the prediction distribution of the model generating the t-th token be denoted as p(y|y). <t ,x),

[0018] Among them, y <t ={y1,y2,…,y t-1} represents the first t-1 tokens generated by the model;

[0019] Write text with the same semantics as input x.

[0020] Furthermore, given input x, a semantically preserving intervention method is used to generate L texts that are semantically identical to x but express different meanings.

[0021] The model outputs y based on the original input x. * and the predicted distribution of each token

[0022]

[0023] in

[0024] y * With each perturbation of the input By concatenating the inputs into the model, the predicted distribution of each output token after perturbation is obtained.

[0025] The change in the token prediction distribution before and after the average perturbation is used as the uncertainty estimate score.

[0026] Furthermore, the step of using the average change in the token prediction distribution before and after the perturbation as the uncertainty estimate score specifically involves:

[0027]

[0028] Among them, D H Let α be a measure of the difference between two probability distributions. t It is information weight.

[0029] A large model uncertainty estimation system based on semantically preserved perturbation, the system employing the aforementioned large model uncertainty estimation method based on semantically preserved perturbation, the system comprising:

[0030] Semantic intervention module: Semantic preservation intervention is performed by rewriting large models or using SOC input;

[0031] Uncertainty scoring module: Calculates uncertainty scores based on semantic preservation intervention;

[0032] Large model uncertainty estimation module: Based on the calculated score, it realizes large model uncertainty estimation based on semantically preserved perturbation.

[0033] Furthermore, the uncertainty scoring calculation module works as follows: Let the original input of the model be x, and the output be y = {y1, y2, ..., y...}. T},

[0034] Where y1, y2, ..., y T The token generated for the model;

[0035] Let the prediction distribution of the model generating the t-th token be denoted as p(y|y). <t ,x),

[0036] Among them, y <t ={y1,y2,…,y t-1} represents the first t-1 tokens generated by the model;

[0037] Write text with the same semantics as input x.

[0038] Furthermore, given input x, a semantically preserving intervention method is used to generate L texts that are semantically identical to x but express different meanings.

[0039] The model outputs y based on the original input x. * and the predicted distribution of each token

[0040]

[0041] in

[0042] y * With each perturbation of the input By concatenating the inputs into the model, the predicted distribution of each output token after perturbation is obtained.

[0043] The change in the token prediction distribution before and after the average disturbance is used as the uncertainty estimate score;

[0044] Specifically, the change in the average token prediction distribution before and after the perturbation is used as the uncertainty estimate score.

[0045]

[0046] Among them, D H Let α be a measure of the difference between two probability distributions. t It is information weight.

[0047] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method described above.

[0048] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0049] The beneficial effects of this invention are:

[0050] This invention actually estimates the expected difference in output distribution before and after the perturbation, which can effectively distinguish between cognitive uncertainty and accidental uncertainty.

[0051] Unlike previous methods, this invention avoids the need to reconstruct a large and complex output space, thereby achieving higher stability and efficiency.

[0052] This invention quantifies the average change in the prediction distribution of the same output token before and after semantic preservation intervention. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the structure of the present invention.

[0054] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0055] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0056] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0057] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0058] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0060] Implementation Method 1

[0061] This implementation provides a method for estimating large model uncertainty based on semantically preserved perturbations, such as... Figures 1-2As shown, the estimation method includes the following steps:

[0062] Step 1: Use large model rewriting or SOC input to perform semantic preservation intervention;

[0063] Step 2: Based on the semantic preservation intervention in Step 1, calculate the uncertainty score;

[0064] Step 3: Based on the score calculated in Step 2, perform large model uncertainty estimation based on semantically preserved perturbation.

[0065] Furthermore, step 1 specifically involves inputting the input portion that needs to be input into the large model into another independent rewritten model;

[0066] If the input is in Chinese, the model is rewritten to generate multiple texts with the same semantics but different expressions.

[0067] If the input is in English, words are randomly selected from the input text. One or more English characters are randomly deleted from each selected word after SOC, and then the English input is obtained.

[0068] Experiments based on semantic similarity demonstrate that both methods can effectively disrupt the surface structure of text while maintaining semantic similarity.

[0069] Furthermore, step 2 specifically involves:

[0070] Let the initial input of the model be x, and the output be y = {y1, y2, ..., y3}. T}, where y1, y2, ..., y T The token generated for the model;

[0071] Let the prediction distribution of the model generating the t-th token be denoted as p(y|y). <t ,x),

[0072] Among them, y <t ={y1,y2,…,y t-1} represents the first t-1 tokens generated by the model;

[0073] Write text with the same semantics as input x.

[0074] Furthermore, given input x, a semantically preserving intervention method is used to generate L texts that are semantically identical to x but express different meanings.

[0075] The model outputs y based on the original input x. * and the predicted distribution of each token

[0076]

[0077] in

[0078] y * With each perturbation of the input By concatenating the inputs into the model, the predicted distribution of each output token after perturbation is obtained.

[0079] The change in the token prediction distribution before and after the average perturbation is used as the uncertainty estimate score.

[0080] Furthermore, the step of using the average change in the token prediction distribution before and after the perturbation as the uncertainty estimate score specifically involves:

[0081]

[0082] Among them, D H Let α be a measure of the difference between two probability distributions; here, the Helinger distance is used. t This is an information weight used to penalize tokens with low information density and unimportant value. Here, the entropy of the prediction distribution function is taken as the weight.

[0083] A more specific embodiment is as follows:

[0084] Experiments were conducted on multiple question-answering datasets, including SciQ, TriviaQA, CoQA, AmbigQA, and TruthfulQA. Uncertainty scores were used to predict the correctness of the model's responses, i.e., to determine whether the model's output is reliable.

[0085] Higher prediction accuracy means a score that truly reflects the model's confidence in its output. AUROC, a common evaluation metric for classification tasks, is used as the evaluation standard. Experiments were conducted on two current mainstream large models of different sizes, and the proposed method was compared with four baseline methods: Length Normalized Prediction Entropy (LN-PE), INSIDE, Semantic Entropy, and SAR. LN-PE directly uses Monte Carlo estimation to calculate the output spatial entropy based on the length-normalized sentence log probability. INSIDE utilizes the inherent variations in the semantic embeddings of the sampled outputs to quantify uncertainty. Semantic Entropy considers semantic equivalence and estimates the output spatial entropy after clustering semantically equivalent outputs. SAR is the state-of-the-art method, introducing importance weights to shift attention to more relevant words and sentences, thereby improving the uncertainty score.

[0086] The final experimental results are shown in 1:

[0087]

[0088]

[0089] Extensive experiments on five datasets demonstrate that the algorithm consistently outperforms state-of-the-art methods, proving its effectiveness.

[0090] Implementation Method 2

[0091] This embodiment provides a large model uncertainty estimation system based on semantically preserved perturbations. The system uses the large model uncertainty estimation method based on semantically preserved perturbations described in Embodiment 1. Figure 1 As shown, the system includes:

[0092] Semantic intervention module: Semantic preservation intervention is performed by rewriting large models or using SOC input;

[0093] Uncertainty scoring module: Calculates uncertainty scores based on semantic preservation intervention;

[0094] Large model uncertainty estimation module: Based on the calculated score, it realizes large model uncertainty estimation based on semantically preserved perturbation.

[0095] Furthermore, the uncertainty scoring calculation module works as follows: Let the original input of the model be x, and the output be y = {y1, y2, ..., y...}. T},

[0096] Where y1, y2, ..., y T The token generated for the model;

[0097] Let the prediction distribution of the model generating the t-th token be denoted as p(y|y). <t ,x),

[0098] Among them, y <t ={y1,y2,…,y t-1} represents the first t-1 tokens generated by the model;

[0099] Write text with the same semantics as input x.

[0100] Furthermore, given input x, a semantically preserving intervention method is used to generate L texts that are semantically identical to x but express different meanings.

[0101] The model outputs y based on the original input x. * and the predicted distribution of each token

[0102]

[0103] in

[0104] y * With each perturbation of the input By concatenating the inputs into the model, the predicted distribution of each output token after perturbation is obtained.

[0105] The change in the token prediction distribution before and after the average disturbance is used as the uncertainty estimate score;

[0106] Specifically, the change in the average token prediction distribution before and after the perturbation is used as the uncertainty estimate score.

[0107]

[0108] Among them, D H Let α be a measure of the difference between two probability distributions. t It is information weight.

[0109] Implementation Method 3

[0110] This invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. The memory stores software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory and processor are connected via a bus. Specifically, the processor implements any step in Embodiment 1 by running the computer program stored in the memory.

[0111] It should be understood that, in the embodiments of the present invention, the processor may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0112] Memory may include read-only memory, flash memory, and random access memory, and provides instructions and data to the processor. Some or all of the memory may also include non-volatile random access memory.

[0113] As can be seen from the above, the electronic device provided by the embodiments of the present invention can implement the large model uncertainty estimation method based on semantic-preserving perturbation as described in Embodiment 1 by running a computer program. The proposed algorithm, from a causal perspective, establishes a connection between the uncertainty of the large model and the strength of the causal path controlling its inference process. The intuition is that the deeper the model's understanding of the underlying causal mechanism, the lower the uncertainty of the generated response. Based on this premise, a new method for large model uncertainty estimation (UQ) is proposed. This method quantifies uncertainty by measuring the invariance of the output after applying semantic-preserving interventions to the model inputs (prompts). Specifically, it quantifies the average change in the prediction distribution of the same output token before and after the semantic-preserving intervention.

[0114] It should be understood that if the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods described above can also be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0115] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0116] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0117] It should be noted that the methods and detailed examples provided in the above embodiments can be incorporated into the apparatus and devices provided in the embodiments for mutual reference, and will not be repeated here.

[0118] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0119] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units described above is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0120] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for estimating the uncertainty of a large model based on semantically preserving perturbations, characterized in that, The estimation method includes the following steps: Step 1: Use large model rewriting or SOC input to perform semantic preservation intervention; Step 2: Based on the semantic preservation intervention in Step 1, calculate the uncertainty score; Step 3: Based on the score calculated in Step 2, perform large model uncertainty estimation based on semantically preserved perturbation; Specifically, step 1 involves inputting the input part that needs to be input into the large model into another independent rewritten model. If the input is in Chinese, the model is rewritten to generate multiple texts with the same semantics but different expressions. If the input is in English, words are randomly selected from the input text. One or more English characters are randomly deleted from each selected word after SOC, and then the English input is obtained. Step 2 specifically involves assuming the original input of the model is... The output is , in, The token generated for the model; Generate the model The predicted distribution of each token is written as... , in, It is the pre-model generation One token; Will be with input Semantic text writing ; Given input Generate using semantic preservation intervention methods Individual and Texts with the same meaning but different expressions ; The model is obtained based on the original input. Output and the predicted distribution of each token in ; Will With each perturbation of the input By concatenating the inputs into the model, the predicted distribution of each output token after perturbation is obtained. ; The change in the token prediction distribution before and after the average disturbance is used as the uncertainty estimate score; Specifically, the change in the average token prediction distribution before and after the perturbation is used as the uncertainty estimate score. in, This is a measure of the difference between two probability distributions. It is information weight. .

2. A large model uncertainty estimation system based on semantically preserved perturbation, characterized in that, The system uses the large model uncertainty estimation method based on semantically preserved perturbation as described in claim 1, and the system comprises: Semantic intervention module: Semantic preservation intervention is performed by rewriting the input using a large model; Uncertainty scoring module: Calculates uncertainty scores based on semantic preservation intervention; Large model uncertainty estimation module: Based on the calculated score, it realizes large model uncertainty estimation based on semantically preserved perturbation.

3. The system according to claim 2, characterized in that, The uncertainty scoring calculation module works as follows: assuming the original input of the model is... The output is , in, The token generated for the model; Generate the model The predicted distribution of each token is written as... , in, It is the pre-model generation One token; Will be with input Semantic text writing .

4. The system according to claim 3, characterized in that, Given input Generate using semantic preservation intervention methods Individual and Texts with the same meaning but different expressions ; The model is obtained based on the original input. Output and the predicted distribution of each token in ; Will With each perturbation of the input By concatenating the inputs into the model, the predicted distribution of each output token after perturbation is obtained. ; The change in the token prediction distribution before and after the average disturbance is used as the uncertainty estimate score; Specifically, the change in the average token prediction distribution before and after the perturbation is used as the uncertainty estimate score. in, This is a measure of the difference between two probability distributions. It is information weight. .

5. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method as described in claim 1.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in claim 1.

Images