A causal analysis method and system for prompt-based inquiry
Through the structural causal model formalizes the detection process and blocking the backdoor path using causal intervention technology, the problems of inaccuracy and instability of existing detection methods are solved, and more accurate and reliable causal relationship evaluation is achieved.
Patent Information
- Application Number
- CN202210001713.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-04
- Publication Date
- 2025-05-16
- Estimated Expiration
- 2042-01-04
AI Technical Summary
The existing prompt-based detection methods have problems of inaccuracy, instability and unreliability, resulting in the true capabilities of pre-trained models not being accurately evaluated.
The structural causal model is used to formalize the prompt-based detection process, identify the true causal relationship of the expected evaluation and the backdoor path that leads to deviations in the evaluation results, and block the corresponding backdoor path using causal intervention technology.
Effectively identify, understand and eliminate deviations in the detection process, and obtain stable, accurate and reliable causal evaluation results.
Smart Images
Figure CN114492806B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a causal analysis method and system for prompt-based evidence discovery, and belongs to the field of natural language processing. Background Art
[0002] With the great success of large-scale pre-trained models in the field of natural language processing, many studies have focused on probing what knowledge is contained in existing pre-trained models. Prompt-based probing is one of the most widely used probing methods: by constructing task-specific prompts to ask questions to the pre-trained model, the model's knowledge on this task is evaluated. For example, to evaluate whether the pre-trained model knows the birthday of "Michael Jordan", we can input "Michael Jordan was born in [MASK]" to the pre-trained model, where "Michael Jordan" is the query input in natural language, "was born in" is the prompt in natural language, and "[MASK]" is the placeholder for the output prediction result of the pre-trained model. In recent years, researchers have constructed multiple evaluation datasets (such as LAMA, LM diagnostics, X-FACTE, BioLAMA, etc.) that use prompts to probe the knowledge of pre-trained models, and regard the performance of the model on these datasets as its ability on the corresponding tasks.
[0003] However, many studies have found that the current paradigm of using prompts for verification is inaccurate, unstable, and unreliable. The biases in these verification processes will make it impossible to accurately evaluate the true capabilities of the pre-trained model, misleading our understanding of the pre-trained model and even leading to wrong decisions. Therefore, in order to accurately evaluate the task-specific capabilities of the pre-trained model, it is urgent to answer three core questions: 1) What are the biases in the existing prompt-based verification paradigm? 2) Where do these biases come from? 3) How to eliminate these biases? Summary of the invention
[0004] In order to solve the above problems, the present invention uses a structural causal model to formalize the prompt-based exploration process, distinguishes the true causal relationship of the desired evaluation and the backdoor path that causes the evaluation result deviation, and uses causal intervention technology to block the corresponding backdoor path, thereby achieving the purpose of eliminating the deviation in the evaluation process.
[0005] The technical solution adopted by the present invention is as follows:
[0006] A causal analysis method for prompt-based heuristics includes the following steps:
[0007] Establish a structural causal model to formalize the interaction between various variables in the prompt-based inquiry process;
[0008] Based on the structural causal model, the real causal relationship expected to be evaluated and the backdoor path that confuses the evaluation results in the verification process are identified, and the deviation caused by the backdoor path is analyzed;
[0009] The backdoor criterion is used to block the backdoor path that causes bias and obtain unbiased causal relationship evaluation results.
[0010] Furthermore, the structural causal model includes multiple key variables, which describe the causal relationship among four key processes in prompt-based exploration: model pre-training, prompt selection, natural language test set generation, and performance evaluation.
[0011] Furthermore, the structural causal model contains 11 key variables: pre-training corpus distribution D a ; pre-training corpus C; pre-training model M; language expression distribution L; task R; prompt P; task-specific predictor I; test data distribution D b ; Sampled test data T; Natural language test data X; Evaluation performance E.
[0012] Furthermore, the backdoor paths for confusing the evaluation results are three backdoor paths, which reflect the pseudo-correlation between the pre-trained model M and the evaluation performance E, thereby leading to three deviations: prompt preference deviation, instance natural language deviation, and sampling difference deviation.
[0013] Furthermore, the manifestations of the prompt preference bias include: for the same pre-trained model, when prompts with the same semantics but different expressions are used for evaluation, the evaluation results show obvious instability, and on the same task, when prompts with the same semantics but different expressions are used, the rankings between models show inconsistency, and therefore reliable evaluation results cannot be obtained; the manifestations of the instance natural language deviation include: for the same test example, different natural language expressions will lead to inconsistent test results; the manifestations of the sampling difference deviation include: the performance differences between different pre-trained models depend not only on the differences in their corresponding task capabilities, but also on the differences between the distributions of the pre-training data and the test data sampled from different distributions bound to them.
[0014] Furthermore, the backdoor path that causes the deviation is blocked by using the backdoor criterion, which is to find a variable set Z, which satisfies the condition that it does not contain descendant nodes of the pre-trained model M, and Z blocks every path between the pre-trained model M and the evaluation performance E that contains a path pointing to M.
[0015] A causal analysis system for prompt-based forensics using the above method comprises:
[0016] Structural causal model building module, used to build structural causal models to formalize the interaction between various variables in the prompt-based inquiry process;
[0017] The deviation analysis module is used to identify the true causal relationship expected to be evaluated and the backdoor path that confuses the evaluation results during the investigation process based on the structural causal model, and analyze the deviation caused by the backdoor path;
[0018] The causal intervention module is used to block the backdoor path that causes bias using the backdoor criterion and obtain unbiased causal relationship evaluation results.
[0019] The beneficial effects of the present invention are as follows:
[0020] The present invention uses a structural causal model to analyze prompt-based evidence collection, which can effectively identify, understand and eliminate the deviations in the evidence collection process, and obtain stable, accurate and reliable causal relationship evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 It is a schematic diagram of the structural causal model and the three existing backdoor paths.
[0022] Figure 2 This is a schematic diagram of the performance differences caused by prompt preference bias.
[0023] Figure 3 This is a schematic diagram of the unstable pre-trained model ranking caused by prompt preference bias.
[0024] Figure 4 This is a schematic diagram of the inconsistent model predictions caused by the natural language deviation of the instance. DETAILED DESCRIPTION
[0025] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific embodiments and drawings.
[0026] In order to identify, understand and eliminate the bias in the existing prompt-based inquiry paradigm, the present invention provides a causal analysis framework for prompt-based inquiry. Figure 1 The main analysis results are presented, including: (1) a structural causal model that formalizes the interaction between variables in the prompt-based inquiry process; (2) an analytical framework for biases in the existing inquiry process based on the structural causal model; and (3) a causal intervention method based on the backdoor criterion to eliminate biases in the inquiry process.
[0027] The present invention provides a causal analysis method for prompt-based evidence collection, the key steps of which include:
[0028] 1) Formalize the structural causal model of the interaction between variables in the prompt-based discovery process. The structural causal model contains 11 key variables and describes the causal relationship in the four key processes of model pre-training, prompt selection, natural language test set generation, and performance evaluation in prompt-based discovery.
[0029] 2) An analytical framework for the biases in the existing verification process based on the above structural causal model. Based on the structural causal model, the true causal relationship of the expected evaluation and three backdoor paths that confuse the evaluation results are identified, and the three biases caused by the three backdoor paths are analyzed: prompt preference bias, instance natural language bias, and sampling difference bias.
[0030] 3) A causal intervention method based on the backdoor criterion to eliminate the bias of the verification process. Based on the analytical framework of 2), the backdoor criterion is used to block the three backdoor paths that cause the bias, and an unbiased causal relationship evaluation result can be obtained.
[0031] The present invention relates to the following key elements:
[0032] 1. Formalize the structural causal model of the interaction between various variables in the prompt-based inquiry process.
[0033] Structural causal models describe the interaction between related variables in a system. Each structural causal model is associated with a corresponding graphical causal model G = {V, f}, where the nodes in the graph represent the variables V in the structural causal model, and the edges in the graph describe the causal relationships f between the variables. Figure 1 As shown in (a), the interaction relationship between various variables in the prompt-based inquiry process is formalized using a structural causal model, which includes 11 key variables:
[0034] (1) Pre-training corpus distribution D a ;
[0035] (2) Pre-training corpus C, for example, the pre-training corpus of GPT2 is WebText;
[0036] (3) Pre-trained model M;
[0037] (4) Language expression distribution L, which can affect the process of natural language conversion of a concept, such as task to prompt, entity to name, etc.;
[0038] (5) Task R, such as extracting knowledge related to birthplace;
[0039] (6) Prompt P. For example, “x was born in y”;
[0040] (7) Task-specific predictor I, e.g., by combining the prompt “x was born in y”, BERT becomes a predictor of birthplace;
[0041] (8) Test data distribution D b ;
[0042] (9) Sampled test data T, for example, the entity pair corresponding to the relation “birthplace” in Wikidata<Q41421,Q18419> ;
[0043] (10) Natural language test data X, for example<Q41421,Q18419> Natural language<MichaelJordan,Brooklyn> ;
[0044] (11) The evaluation performance E is calculated by the prediction results of predictor I on X.
[0045] This structural causal model describes the model pre-training in prompt-based exploration {D a ,L}→C→M, prompt selection {R,L}→P, natural language test set generation {D b ,R}→T→X←L and performance evaluation {M,P}→I→E←X.
[0046] 2. An analytical framework for the deviations in the existing exploration process based on the above structural causal model.
[0047] When exploring the ability of a pre-trained model in a specific task, what we really want to evaluate is the true causal effect of the model M on the performance E under the condition that the task R = r. In causal theory, we can use Indicates. Among them, is a probability label, do is an operator in causal intervention, m is a specific model, and r is a specific task. This causal effect is represented by the path M→I→E in the structural causal model constructed by the present invention. However, it can be clearly seen from Figure 1 In (a), we identify three backdoor paths from model M to performance E, such as Figure 1 As shown in (b)-(d) of Figure 2, these three backdoor paths will lead to a pseudo-correlation between the model M and the performance E, which in turn leads to three deviations in the verification process, including:
[0048] (1) Prompt preference bias: It comes from the backdoor path M←C←L→P→I→E, which will cause the evaluation performance to be affected by the model's preference for different prompts, making it impossible to truly evaluate the model's capabilities. Its specific manifestations include, Figure 2For example, for the same pre-trained model, the evaluation results show obvious instability when using prompts with the same semantics but different expressions; and for the same task, the rankings between models show inconsistency when using prompts with the same semantics but different expressions, such as Figure 3 Therefore, no reliable evaluation results can be obtained.
[0049] (2) Instance natural language deviation: This is caused by the backdoor path M←C←L→X→E, which will cause the evaluation performance to be confused by the pseudo-correlation brought by the backdoor path, thus failing to truly evaluate the model's capabilities. Specifically, for the same test example, different natural language expressions will lead to inconsistent test results. Figure 4 A quantitative analysis of this phenomenon is presented.
[0050] (3) Sampling difference bias: from the backdoor path Where D a and D b The causal relationship between them varies in different pre-trained models and evaluation data sets, but no matter what the causal relationship is, the backdoor path will exist. Specifically, the performance difference between different pre-trained models depends not only on the ability difference of their corresponding tasks, but also on the difference between the distribution of pre-training data and test data sampled from different distributions bound to them.
[0051] 3. A causal intervention method based on the backdoor criterion to eliminate bias in the inquiry process.
[0052] To eliminate the three deviations mentioned above, we only need to block the three corresponding backdoor paths to calculate the true causal effect from model M to performance E. The backdoor criterion is a common tool used to achieve this goal. According to the backdoor criterion, we only need to find a variable set Z that does not contain descendant nodes of M and blocks every path between M and E that points to M. Figure 1 For the structural causal model in (a), we can choose Z = {X, P}, then the true causal effect of model M to performance E can be calculated by the following formula:
[0053]
[0054] Among them, p represents a specific prompt, x represents natural language data, represents the joint distribution of prompt p and natural language data x, represents the evaluation results of model m under task r using prompt p and natural language data x.
[0055] Through the above-mentioned causal intervention based on the backdoor criterion, the bias existing in the prompt-based exploration process can be eliminated, and a more accurate, stable and reliable evaluation result can be obtained.
[0056] The following is an example of evaluating the factual knowledge of the pre-trained model related to the relationship "birthplace" in factual knowledge exploration to illustrate the method of the present invention. The method includes the following steps:
[0057] (I) Constructing a structural causal model corresponding to factual knowledge exploration, which is structurally similar to Figure 1 a, we only need to instantiate task R as the relationship “birthplace” and instantiate the evaluation dataset as the set of entity pairs corresponding to the relationship “birthplace”.
[0058] (ii) Based on the above structural causal model, we found that all three backdoor paths exist, that is, there are three biases in factual knowledge exploration: prompt preference bias, instance natural language bias, and sampling difference bias.
[0059] (III) Select a set Z = {X, P} to block the backdoor path. Specifically, sample the relationship "birthplace" to obtain multiple prompts with the same semantics but different expressions, such as "x was born in y" and "The birthplace of xis y", etc. For each entity in the test set, sample multiple corresponding natural language expressions, such as "IsaacNewton" and "Sir Isaac Newton", etc. Then calculate the evaluation result according to the following formula:
[0060]
[0061] More accurate, stable and reliable evaluation results can be obtained.
[0062] Based on the same inventive concept, another embodiment of the present invention provides a causal analysis system for prompt-based forensics using the above method, which includes:
[0063] Structural causal model building module, used to build structural causal models to formalize the interaction between various variables in the prompt-based inquiry process;
[0064] The deviation analysis module is used to identify the true causal relationship expected to be evaluated and the backdoor path that confuses the evaluation results during the investigation process based on the structural causal model, and analyze the deviation caused by the backdoor path;
[0065] The causal intervention module is used to block the backdoor path that causes bias using the backdoor criterion and obtain unbiased causal relationship evaluation results.
[0066] Based on the same inventive concept, another embodiment of the present invention provides an electronic device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.
[0067] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, CD), which stores a computer program. When the computer program is executed by a computer, it implements the various steps of the method of the present invention.
[0068] In other embodiments, only the structural causal model needs to be modified accordingly, and the subsequent process remains unchanged, and the analysis system of the present invention can be applied to optimizable prompts and all evaluations of pre-trained models.
[0069] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.
Claims
1. A causal analysis method for prompt-based forensics, characterized in that: The following steps are involved: Establish a structural causal model to formalize the interaction between various variables in the prompt-based inquiry process; Based on the structural causal model, the real causal relationship expected to be evaluated and the backdoor path that confuses the evaluation results in the verification process are identified, and the deviation caused by the backdoor path is analyzed; The backdoor criterion is used to block the backdoor path that causes bias and obtain unbiased causal relationship evaluation results; The structural causal model contains 11 key variables: pre-training corpus distribution D a ; pre-training corpus C; pre-training model M; language expression distribution L; task R; prompt P; task-specific predictor I; test data distribution D b ; Sampled test data T; Natural language test data X; Evaluation performance E; The backdoor paths for confusing the evaluation results are three backdoor paths, which reflect the pseudo-correlation between the pre-trained model M and the evaluation performance E, thereby leading to three deviations: prompt preference deviation, instance natural language deviation, and sampling difference deviation.
2. The method according to claim 1, characterized in that The structural causal model contains multiple key variables, describing the causal relationship among the four key processes of model pre-training, prompt selection, natural language test set generation, and performance evaluation in prompt-based exploration.
3. The method according to claim 1, characterized in that The manifestations of the prompt preference bias include: for the same pre-trained model, when prompts with the same semantics but different expressions are used for evaluation, the evaluation results show obvious instability, and on the same task, when prompts with the same semantics but different expressions are used, the rankings between models show inconsistency, and therefore reliable evaluation results cannot be obtained; the manifestations of the instance natural language deviation include: for the same test sample, different natural language expressions will lead to inconsistent test results; the manifestations of the sampling difference deviation include: the performance differences between different pre-trained models depend not only on the differences in their corresponding task capabilities, but also on the differences between the distributions of the pre-training data and the test data sampled from different distributions bound to them.
4. The method according to claim 1, characterized in that: The method of using the backdoor criterion to block the backdoor path that causes the deviation is to find a variable set Z that satisfies the descendant node that does not contain the pre-trained model M, and Z blocks every path between the pre-trained model M and the evaluation performance E that contains a path pointing to M.
5. The method according to claim 4, characterized in that According to the structural causal model, we select Z = {X, P}, and the true causal effect from the pre-trained model M to the evaluation performance E is calculated by the following formula: Among them, p represents a specific prompt, x represents natural language data, represents the joint distribution of prompt p and natural language data x, represents the evaluation results of model m under task r using prompt p and natural language data x.
6. A causal analysis system for prompt-based forensics using the method of any one of claims 1 to 5, characterized in that: include: Structural causal model building module, used to build structural causal models to formalize the interaction between various variables in the prompt-based inquiry process; The deviation analysis module is used to identify the true causal relationship expected to be evaluated and the backdoor path that confuses the evaluation results during the investigation process based on the structural causal model, and analyze the deviation caused by the backdoor path; The causal intervention module is used to block the backdoor path that causes bias using the backdoor criterion and obtain unbiased causal relationship evaluation results.
7. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 5 is implemented.