Prompt sensitivity analysis method for large language model

Abstract

The invention relates to a big language model cue word sensitivity analysis method, which is characterized by comprising the following steps: S1, obtaining instance data, each instance corresponding to a cue word set, the cue word set comprising a plurality of topic cue words, and inputting the instance data into a to-be-detected big language model, an answer analysis cue word sensitivity analysis index PSS based on a large language model; s2, inputting the multi-topic cue word in the instance data into a to-be-detected large language model, calculating a decoding confidence coefficient according to the probability of a maximum probability token corresponding to the multi-topic cue word predicted by the large language model, and calculating an average decoding confidence coefficient based on the decoding confidence coefficient of the multi-topic cue word corresponding to a plurality of instances; and S3, analyzing the overall cue word sensitivity of the to-be-detected large language model based on the cue word sensitivity analysis index PSS and the average decoding confidence coefficient. Compared with the prior art, the method has the advantages of improving the accuracy of sensitivity analysis of the cue words of the large language model and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of prompt word sensitivity of a large language model, and in particular to a prompt word sensitivity analysis method of a large language model. Background Art

[0002] Large language models are very sensitive to slight changes in prompt words. Some slight changes, such as adding a few spaces, can also significantly change the performance of large language models. This sensitivity to prompt words makes it difficult for large language model developers to determine whether the change in the model's score on the test set is due to the improvement of model performance or the choice of prompt words when evaluating the model; in addition, when using the model, users often need to tune the prompt words multiple times to obtain higher quality output.

[0003] Traditional prompt word sensitivity analysis techniques mostly analyze the prompt word sensitivity of the model from the dataset level, focusing on the overall change trend of model performance under different prompt formats. For example, the PromptBench benchmark system evaluates the performance of several mainstream large language models under different prompt words. Pezeshkpour et al. found that even minor grammatical or lexical changes can lead to a significant drop in the performance of large language models. These studies reveal the fact that large language models are highly sensitive to prompts, laying the foundation for further understanding and alleviating this problem.

[0004] However, existing methods mainly focus on analyzing cue sensitivity at the dataset level, mostly based on the score changes of different cue word templates on the same dataset. This evaluation method is not accurate and cannot provide feedback on the model's true cue word sensitivity. In addition, existing analysis of large language models does not involve subjective evaluation and lacks consideration of actual application scenarios and user needs. Existing work has also not explored the root cause of large language models' sensitivity to cues, limiting our understanding of their underlying mechanisms. Summary of the invention

[0005] The purpose of the present invention is to provide a cue word sensitivity analysis method for a large language model in order to improve the accuracy of the cue word sensitivity analysis for a large language model.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A cue word sensitivity analysis method for a large language model, the method comprising the following steps:

[0008] S1. Obtaining instance data, wherein the instance data includes multiple instances, each instance corresponds to a prompt word set, and the prompt word set includes multiple-choice prompt words, inputting the instance data into a large language model to be tested, and analyzing the prompt word sensitivity analysis index PSS based on the answer of the large language model;

[0009] S2, inputting the multiple-choice prompt words in the instance data into the large language model to be detected, calculating the decoding confidence according to the probability of the maximum probability token corresponding to the multiple-choice prompt words predicted by the large language model, and calculating the average decoding confidence based on the decoding confidences of the multiple-choice prompt words corresponding to multiple instances;

[0010] S3. Analyze the overall cue word sensitivity of the large language model to be tested based on the cue word sensitivity analysis index PSS and the average decoding confidence.

[0011] Furthermore, the prompt word sensitivity analysis index PSS is the average value of the sensitivity of all instances in the instance data.

[0012] Furthermore, the sensitivity is:

[0013]

[0014] in, is the sensitivity of an instance, Y(p) represents the performance index under the prompt word p, C(|P|,2) represents the number of prompt word pairs in the same instance, and i and j represent two different prompt word indexes.

[0015] Furthermore, the performance indicator under the prompt word p is the correctness of the answer of the large language model when a given true value is included in the instance data.

[0016] Furthermore, the correctness of the language model answer is the similarity between the large language model answer and a given true value.

[0017] Furthermore, the performance indicator under the prompt word p is the score of the large language model answer when the given true value is not included in the instance data.

[0018] Furthermore, the decoding confidence corresponding to an instance is:

[0019]

[0020] Among them, Probability(t next |p) represents the probability of the maximum probability token predicted by the model under the prompt word p in the prompt word set P.

[0021] Furthermore, in the process of analyzing the overall prompt word sensitivity prompt words of the large language model to be tested, the sensitivity analysis index PSS represents the probability of inconsistency in correctness between any two prompts in the same instance in subjective and objective evaluation.

[0022] Furthermore, in the process of analyzing the overall prompt word sensitivity prompt words of the large language model to be tested, the sensitivity analysis index PSS represents the difference in average answer quality between two prompts in the same instance in objective evaluation.

[0023] Furthermore, the average decoding confidence is positively correlated with the cue word sensitivity.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention provides an indicator PPS for measuring the prompt word sensitivity of a large language model from an instance level, so that a more comprehensive and detailed analysis of the prompt word sensitivity of a large language model can be performed, and the prompt word sensitivity of a large language model can be analyzed in objective and subjective evaluations. At the same time, the prompt word sensitivity mechanism of the large language model is analyzed, and a methodology for analyzing the prompt word sensitivity from the perspective of the decoding confidence of the large language model is proposed, thereby improving the accuracy of the prompt word sensitivity analysis of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of the present invention;

[0027] Figure 2 This is a graph showing the results of a prompt word sensitivity difference experiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0029] This paper introduces the ProSA framework, focuses on instance-level analysis, and proposes a cue word sensitivity analysis method for a large language model. The flowchart of the method is as follows: Figure 1 As shown, the present invention quantifies the average difference in the responses of LLMs to different prompt variants of the same instance, and uses decoding confidence to reveal the underlying mechanism of prompt sensitivity. The present invention aims to comprehensively evaluate and analyze the prompt sensitivity of large language models and reveal the mechanism behind it. ProSA emphasizes instance-level analysis, includes a novel sensitivity metric PSS, and is able to analyze the prompt word sensitivity of large language models in objective and subjective evaluations. In addition, this framework also uses decoding confidence to explore the root causes of prompt sensitivity, in order to provide guidance for the development of large language models that are more robust and closer to user needs.

[0030] The method of the present invention comprises the following steps:

[0031] S1. Obtaining instance data, wherein the instance data includes multiple instances, each instance corresponds to a prompt word set, and the prompt word set includes multiple-choice prompt words, inputting the instance data into a large language model to be tested, and analyzing the prompt word sensitivity analysis index PSS based on the answer of the large language model;

[0032] S2, inputting the multiple-choice prompt words in the instance data into the large language model to be detected, calculating the decoding confidence according to the probability of the maximum probability token corresponding to the multiple-choice prompt words predicted by the large language model, and calculating the average decoding confidence based on the decoding confidences of the multiple-choice prompt words corresponding to multiple instances;

[0033] S3. Analyze the overall cue word sensitivity of the large language model to be tested based on the cue word sensitivity analysis index PSS and the average decoding confidence.

[0034] The present invention designs an instance-level prompt word sensitivity analysis index PSS, which is specifically composed as follows:

[0035]

[0036] For a prompt word p in the prompt word set P for a specific instance, Y(p) represents the performance indicator under this prompt p. For an instance with a given true value, Y(p) refers to the correctness of the answer given by the large language model. For tasks without a clear true value, the answer usually has a score indicating the quality of generation, and Y(p) refers to this score, which ranges between [0, 1]. |Y(p i )-Y(p j )|Represents prompt p i and prompt p j The absolute difference between the performance indicators. C(|P|,2) represents the number of prompt pairs in the same instance. PSS is the absolute difference between the performance indicators of all instances in the same dataset. The average value of .

[0037] Due to the differences in task types and evaluation methods, PSS has different meanings in objective and subjective evaluations. In objective evaluations, PSS represents the probability of inconsistency between any two prompts in the same instance. In subjective evaluations, PSS represents the difference in average answer quality between two prompts in the same instance.

[0038] Compared with statistical analysis at the dataset level, PSS provides a more accurate and intuitive representation of cue sensitivity.

[0039] In addition, the present invention also uses token probabilities to calculate the decoding confidence of the large language model in the multiple-choice question setting. The decoding confidence in an example is defined as follows:

[0040]

[0041] Here, Probability(t next |p) represents the probability of the maximum probability token predicted by the model under the prompt word p in the prompt word set P. The average decoding confidence of the large language model is The average decoding confidence in the present invention is positively correlated with the cue word sensitivity.

[0042] The key point of the present invention is the framework of the prompt word sensitivity analysis of the large language model, which can be specifically divided into the following two points:

[0043] (1) The present invention provides an indicator PPS that measures the prompt word sensitivity of a large language model from an instance level.

[0044] This enables a more comprehensive and detailed analysis of the cue word sensitivity of large language models.

[0045] The present invention analyzes the cue word sensitivity mechanism of a large language model for the first time, and proposes a methodology for analyzing the cue word sensitivity from the perspective of decoding confidence of a large language model.

[0046] The present invention has the following advantages:

[0047] (1) Compared with the cue word sensitivity analysis index at the dataset level, the instance-level analysis index PSS of the present invention can more comprehensively and meticulously characterize the cue word sensitivity of a large language model.

[0048] This invention is the first to analyze the cue word sensitivity of a large language model from the perspective of interpretability. The invention conducts experiments on four objective datasets and two subjective datasets.

[0049] First, on the four datasets of CommonsenseQA[3], ARC-Challenge[4], MATH[5], and HumanEval[6], we used eight models, including the InternLM2 series[7], Llama3 series[8], Qwen1.5 series[9], and Mistral-7B

[10] , to calculate the PSS on 12 prompt words. The experimental results show that there are large differences in prompt word sensitivity between different models. The results are as follows: Figure 2 shown.

[0050] In addition, we also conducted experiments on two subjective datasets, LC AlpacaEval 2.0

[11] and Arena Hard Auto

[12] , using five large language models on three sets of prompt words. The results are shown in Table 1:

[0051] Table 1 Experimental results

[0052] Model LC AlpacaEval 2.0 Arena Hard Auto InternLM2-20B-chat 0.022 0.249 Llama3-8B-instruct 0.013 0.266 Llama3-70B-instruct 0.016 0.258 Qwen1.5-14B-chat 0.022 0.249 Qwen1.5-72B-chat 0.036 0.250

[0053] It can be seen that different models show different sensitivities to cue words on different subjective benchmarks, and are more sensitive to cue words on the ArenaHard Auto benchmark.

[0054] The literature cited in the above experiment is as follows:

[0055] 【1】Promptbench:Towards evaluating the robustness of large language models on adversarial prompts

[0056] 【2】Large language models sensitivity to the order of options in multiple-choicequestions

[0057] 【3】Commonsenseqa: A question answering challenge targeting commonsenseknowledge

[0058] 【4】Think you have solved question answering? try arc, the ai2 reasoningchallenge.【5】Measuring mathematical problem solving with the math dataset

[0059] 【6】Evaluating Large Language Models Trained on Code

[0060] 【7】Internlm2 technical report

[0061] 【8】Llama 3Model Card

[0062] 【9】Qwen Technical Report

[0063]

[10] Mistral 7B

[0064]

[11] Length-Controlled AlpacaEval:A Simple Way to Debias AutomaticEvaluators

[12] From Live Data to High-Quality Benchmarks:The Arena-HardPipeline

[0065] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A cue word sensitivity analysis method for a large language model, characterized in that: The method comprises the following steps: S1. Obtaining instance data, wherein the instance data includes multiple instances, each instance corresponds to a prompt word set, and the prompt word set includes multiple-choice prompt words, inputting the instance data into a large language model to be tested, and analyzing the prompt word sensitivity analysis index PSS based on the answer of the large language model; S2, inputting the multiple-choice prompt words in the instance data into the large language model to be detected, calculating the decoding confidence according to the probability of the maximum probability token corresponding to the multiple-choice prompt words predicted by the large language model, and calculating the average decoding confidence based on the decoding confidences of the multiple-choice prompt words corresponding to multiple instances; S3. Analyze the overall cue word sensitivity of the large language model to be tested based on the cue word sensitivity analysis index PSS and the average decoding confidence.

2. The method for analyzing the sensitivity of prompt words of a large language model according to claim 1, characterized in that: The prompt word sensitivity analysis index PSS is the average value of the sensitivity of all instances in the instance data.

3. The method for analyzing the sensitivity of prompt words of a large language model according to claim 2, characterized in that: The sensitivities are: in, is the sensitivity of an instance, Y(p) represents the performance index under the prompt word p, C(|P|,2) represents the number of prompt word pairs in the same instance, and i and j represent two different prompt word indexes.

4. The method for analyzing the sensitivity of prompt words of a large language model according to claim 3, characterized in that: The performance indicator under the prompt word p is the correctness of the answer given by the large language model when the true value is included in the instance data.

5. The method for analyzing the sensitivity of prompt words of a large language model according to claim 4, characterized in that: The correctness of the language model answer is the similarity between the large language model answer and the given true value.

6. The method for analyzing the sensitivity of prompt words of a large language model according to claim 3, characterized in that: The performance indicator under the prompt word p is the score of the large language model answer when the given true value is not included in the instance data.

7. The method for analyzing the sensitivity of prompt words of a large language model according to claim 1, characterized in that: The decoding confidence corresponding to an instance is: Among them, Probability(t next |p) represents the probability of the maximum probability token predicted by the model under the prompt word p in the prompt word set P.

8. The method for analyzing the sensitivity of prompt words of a large language model according to claim 1, characterized in that: In the process of analyzing the overall prompt word sensitivity of the large language model to be tested, the sensitivity analysis index PSS represents the probability of inconsistency in correctness between any two prompts in the same instance in subjective and objective evaluation.

9. The method for analyzing the sensitivity of prompt words of a large language model according to claim 1, characterized in that: In the process of analyzing the overall prompt word sensitivity of the large language model to be tested, the sensitivity analysis indicator PSS represents the difference in average answer quality between two prompts in the same instance in objective evaluation.

10. The method for analyzing the sensitivity of prompt words of a large language model according to claim 1, characterized in that: The average decoding confidence is positively correlated with the cue word sensitivity.

Images