A large model evaluation method and device based on adversarial attacks

Abstract

The application provides a large model evaluation method and device based on an adversarial attack, and relates to the technical field of artificial intelligence, and the method comprises the following steps: analyzing the confounding effect of a confounding factor on a large model prediction result through a confounding path by using a pre-constructed structural causal model; based on the analysis result of the structural causal model, key samples affected by the confounding effect are screened by comparing the output differences of different prompts; for the key samples, an adversarial sample is generated by a black box method or a white box method; and the large model is evaluated by using the adversarial sample. The application analyzes the confounding effect of the large model, reduces the influence of the confounding factor through the causal theory, and thus finds the key samples in the data set; for the key samples, an adversarial sample generation method in the black box and white box scenarios is proposed, and the adversarial sample is used for large model evaluation, which can more effectively evaluate the robustness of the large model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a large model evaluation method and device based on adversarial attacks. BACKGROUND

[0002] Large models represented by ChatGPT have breakthrough natural language understanding and generation capabilities, but the training and application of large models are still in the early stages and face major bottlenecks. Whether it is self-training large models, domain adaptation and fine-tuning based on open-source large models, or commercial large models, they all face the key problem of how to evaluate the intelligence level of large models. Rapid and real intelligence level evaluation not only supports agile parameter optimization and version iteration to improve training efficiency, but also has important value for large model technology selection and credibility assurance.

[0003] The evaluation method based on existing data sets evaluates the performance of the model based on a fixed set of test data. This method can help us understand the overall performance of the large model to some extent. However, due to the complex internal mechanism of the pre-trained large model and the characteristics of the high-dimensional input space, traditional testing methods often cannot fully cover the performance of the model under all possible inputs. Moreover, the testing method based on existing data sets often cannot effectively identify the specific vulnerabilities or weak links of the model.

[0004] Therefore, how to provide automatic and systematic large model intelligence level test evaluation data and evaluation methods is a major practical problem that needs to be solved. SUMMARY

[0005] The present application provides a large model evaluation method and device based on adversarial attacks, which solves the defect that the existing testing method based on existing data sets cannot effectively identify the specific vulnerabilities or weak links of the model, and realizes the automation and systematization of large model intelligence level test evaluation data and evaluation methods.

[0006] The present application provides a large model evaluation method based on adversarial attacks, comprising the following steps.

[0007] Using a pre-constructed structural causal model, analyze the confounding effect of confounding factors on the prediction results of the large model through confounding paths;

[0008] Based on the analysis results of the structural causal model, filter the key samples affected by the confounding effect by comparing the output differences of different prompts;

[0009] For the key samples, generate adversarial samples through black box methods or white box methods;

[0010] Use the adversarial samples to evaluate the large model.

[0011] According to the method for evaluating a large model based on an adversarial attack provided by the application, the construction process of the structural causal model specifically comprises:

[0012] defining variables of the large model and causal relationships between the variables;

[0013] drawing a causal graph DAG based on the causal relationships;

[0014] establishing structural equations of the variables and mathematically describing the causal relationships between the variables.

[0015] According to the method for evaluating a large model based on an adversarial attack provided by the application, the confounding effect of a confounding factor on a prediction result of the large model through a confounding path is analyzed, and the method specifically comprises:

[0016] determining the confounding factor and a confounding path corresponding to the confounding factor through the causal graph DAG;

[0017] eliminating bias through a control variable or intervention to block the confounding path;

[0018] calculating an effect estimation difference before and after the confounding path is blocked through the structural equations, and determining a calculation result of the effect estimation difference as the confounding effect of the confounding factor on the prediction result of the large model through the confounding path.

[0019] According to the method for evaluating a large model based on an adversarial attack provided by the application, the different prompts include simple prompts and complex prompts, wherein the simple prompts only include a task name and a general expression, and the complex prompts include one or more of a detailed task description, a solution step, a potential interference item, a constraint condition and an example;

[0020] Accordingly, based on the analysis result of the structural causal model, key samples affected by the confounding effect are screened by comparing output differences of different prompts, and the method specifically comprises:

[0021] concatenating the same input sample with the simple prompts and the complex prompts respectively to form two input sequences;

[0022] inputting the two input sequences into the large model to generate prediction results, and recording probability distributions of each prediction result;

[0023] calculating a confidence difference of the two prediction results, and if a calculation result of the confidence difference is greater than a preset threshold, the input sample is determined as a key sample.

[0024] According to the method for evaluating a large model based on an adversarial attack provided by the application, the black-box method comprises:

[0025] Replace the i-th position word with a mask to form a new input sample ;

[0026] Determine the importance score of each position of the input sample through the model output:

[0027] ,

[0028] Wherein, is the position importance score of the i-th position of the input sample, is the encoding vector of the i-th position in the input sample, i=1~n, n is the total number of positions of the input sample, is the probability distribution before replacement, is the probability distribution after replacing the i-th position;

[0029] Select the input sample position with the highest score for character-level or semantic-level perturbation.

[0030] The method for evaluating large models based on adversarial attacks provided by the application comprises:

[0031] Generate the optimal perturbation vector based on the gradient-based adversarial sample generation theory ;

[0032] Replace the i-th position word with a mask to form a new input sample ;

[0033] Compare the vector obtained after encoding with the optimal perturbation vector , Select the input sample position with the smallest distance as the optimal perturbation position.

[0034] The method for evaluating large models based on adversarial attacks provided by the application, the adversarial sample comprises the following types:

[0035] Adversarial prompt, inject interference in task instructions;

[0036] Adversarial content, modify key words in the input sample.

[0037] The method for evaluating large models based on adversarial attacks provided by the application, using the adversarial sample, the indicators for evaluating the large model include one or more of the following:

[0038] Accuracy drop, attack success rate, semantic consistency, robustness, prompt sensitivity, semantic understanding and generalization ability, defense ability and causal interpretability.

[0039] The application further provides a large model evaluation device based on an adversarial attack, comprising the following modules.

[0040] The confounding effect analysis module is configured to analyze the confounding effect of confounding factors on the prediction result of the large model through confounding paths by using a pre-constructed structural causal model.

[0041] The key sample screening module is configured to screen key samples affected by the confounding effect by comparing the output difference of different prompts based on the analysis result of the structural causal model.

[0042] The adversarial sample generation module is configured to generate adversarial samples for the key samples by using a black-box method or a white-box method.

[0043] The evaluation module is configured to evaluate the large model by using the adversarial samples.

[0044] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the large model evaluation method based on an adversarial attack according to any one of the above embodiments when executing the computer program.

[0045] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the large model evaluation method based on an adversarial attack according to any one of the above embodiments.

[0046] The application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the large model evaluation method based on an adversarial attack according to any one of the above embodiments.

[0047] The application provides a large model evaluation method and device based on an adversarial attack, which analyzes the confounding effect of confounding factors on the prediction result of the large model through confounding paths by using a pre-constructed structural causal model, screens key samples affected by the confounding effect by comparing the output difference of different prompts based on the analysis result of the structural causal model, generates adversarial samples for the key samples by using a black-box method or a white-box method, and evaluates the large model by using the adversarial samples. The application analyzes the confounding effect of the large model, reduces the influence of confounding factors by using a causal theory, and thus finds key samples in the data set. The application proposes an adversarial sample generation method in a black-box and white-box scenario for the key samples, and uses the adversarial samples for large model evaluation, which can more effectively evaluate the robustness of the large model. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0049] Figure 1 is a flowchart of the method for evaluating a large model based on an adversarial attack provided by the present application.

[0050] Figure 2 is a structural causal model diagram of the information extraction process based on a large model provided by the present application.

[0051] Figure 3 is a flowchart of the generation of an adversarial sample provided by the present application.

[0052] Figure 4 is a structural diagram of the device for evaluating a large model based on an adversarial attack provided by the present application.

[0053] Figure 5 is a structural diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0054] In order to make the objects, technical solutions and advantages of the present application clearer, the following will describe the technical solutions in the present application clearly and completely with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application.

[0055] The present application will be described in detail below with reference to the drawings in the specification. The specific operation methods in the method embodiments can also be applied to the device embodiments or system embodiments. In the description of the present application, unless otherwise specified, "at least one" includes one or more. "Multiple" refers to two or more. For example, at least one of A, B and C includes: A alone, B alone, A and B together, A and C together, B and C together, and A, B and C together. In the present application, " / " means or, for example, A / B can mean A or B; "and / or" in this document only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone.

[0056] The present application will be described in detail below with reference to the drawings in the specification. The specific operation methods in the method embodiments can also be applied to the device embodiments or system embodiments. In the description of the present application, unless otherwise specified, "at least one" includes one or more. "Multiple" refers to two or more. For example, at least one of A, B and C includes: A alone, B alone, A and B together, A and C together, B and C together, and A, B and C together. In the present application, " / " means or, for example, A / B can mean A or B; "and / or" in this document only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone.

[0057] In some embodiments of the present application, as shown in Figure 1 The present application provides a large model evaluation method based on adversarial attacks, comprising the following steps:

[0058] Step 100: Analyze the confounding effect of confounding factors on the prediction results of the large model through confounding paths by using a pre-constructed structural causal model;

[0059] Based on the analysis results of the structural causal model, filter the key samples affected by the confounding effect by comparing the output differences of different prompts;

[0060] Generate adversarial samples through black-box or white-box methods for the key samples;

[0061] Use the adversarial samples to evaluate the large model.

[0062] It should be noted that the existing large model evaluation scheme has a large test dataset, which cannot generate corresponding adversarial samples for each sample, and the evaluation method based on the existing dataset cannot completely cover the performance of the model under all possible inputs, and the test method based on the existing dataset also cannot effectively identify the specific vulnerabilities or weak links of the model.

[0063] Therefore, the present application analyzes the confounding effect of the large model, reduces the influence of confounding factors through causal theory, and finds the key samples in the dataset; proposes an adversarial sample generation method in the black-box and white-box scenarios for the key samples, and uses the generated adversarial samples for large model evaluation, so as to more effectively evaluate the robustness of the large model.

[0064] In some possible embodiments of the present application, the construction process of the structural causal model specifically comprises:

[0065] Define the variables of the large model and the causal relationship between each variable;

[0066] Based on the causal relationship, draw a causal graph DAG;

[0067] Establish a structural equation for each variable to mathematically describe the causal relationship between each variable.

[0068] Specifically, the present embodiment provides a construction method of a structural causal model, which reveals the confounding effect (such as backdoor path interference) of confounding factors such as pre-training knowledge (K) on model output by constructing a structural causal model (Structural Causal Model, SCM).

[0069] Specifically, in the present embodiment, the causal inference theory is introduced, a structural causal model SCM of the bias effect of a large model is proposed, and the formation mode and mechanism of the pre-training knowledge bias effect are explored; on this basis, samples more likely to cause prediction errors of the large model are located for disturbance, and a data set more testing the robustness of the large model is formed.

[0070] In possible embodiments, the structural causal model expresses the causal relationship between key elements in the downstream task process. Taking the classification paradigm as an example, there are five model variables in the SCM:

[0071] Variable 1: Prior knowledge K in the large model;

[0072] Variable 2: Encoder E of the large model;

[0073] Variable 3: Feature representation X generated by the large model;

[0074] Variable 4: Answer generation constraint V (for example, the constraint given in the candidate label set or the prompt provided to the model);

[0075] Variable 5: Final prediction result Y.

[0076] In some possible embodiments of the present application, the confounding effect of the confounding factor on the prediction result of the large model through the confounding path is analyzed, which specifically includes:

[0077] Determine the confounding factor and the confounding path corresponding to the confounding factor through the causal diagram DAG;

[0078] Eliminate bias by controlling variables or intervention to block the confounding path;

[0079] Calculate the difference between the effect estimates before and after blocking the confounding path through the structural equation, and determine the calculation result of the effect estimate difference as the confounding effect of the confounding factor on the prediction result of the large model through the confounding path.

[0080] Specifically, the present embodiment provides an implementation for analyzing confounding effects, and the structural causal model can discover the causal confounding effects caused by confounding factors such as pre-training knowledge.

[0081] In possible embodiments, as shown in Figure 2 In the SCM, the following paths exist:

[0082] V←K→E indicates that the knowledge of the large model directly affects the encoder and the answer generation constraint.

[0083] E→X indicates that the sample features are generated by the encoder.

[0084] X→Y←V means that the final prediction result is determined by the feature and the answer generation constraint together.

[0085] According to the theory of causal inference, Figure 2 There is a backdoor path: X←E←K→V→Y. This path means that there is a confounding factor K in the causal relationship from X to Y. Therefore, when calculating the conditional probability P(Y|X), the influence of K must be considered. That is, the internal knowledge of the large model brings performance improvement, but also plays a role of confounding factor.

[0086] Specifically, the backdoor path: X←E←K→V→Y indicates that the pre-training knowledge (K) as a confounding factor may cause model prediction bias, therefore, the confounding effect needs to be weakened through causal intervention (such as adjusting the prompt constraint V) to locate the samples susceptible to interference.

[0087] From the above analysis, it can be seen that the structural causal model can be used to find the causal confounding effect brought by pre-training knowledge. The direct manifestation of this confounding effect is that it is easy to mislead the output of the large model in some cases. The degree of "easy to mislead" can be reflected by the output result changes brought by simple prompts and complex (or more semantically rich) prompts. On this basis, a means of locating key data samples is provided, that is, perturbing the key data samples.

[0088] In some possible embodiments of the present application, the different prompts include simple prompts and complex prompts, wherein the simple prompts only contain task names and general expressions, and the complex prompts include one or more of detailed task descriptions, solution steps, potential interference items, constraint conditions and examples;

[0089] Correspondingly, based on the analysis result of the structural causal model, the key samples affected by the confounding effect are screened by comparing the output differences of different prompts, specifically including:

[0090] The same input sample is spliced with a simple prompt and a complex prompt respectively to form two input sequences;

[0091] The two input sequences are input into the large model to generate prediction results, and the probability distribution of each prediction result is recorded;

[0092] The confidence difference of the two prediction results is calculated, and if the calculation result of the confidence difference is greater than a preset threshold, the input sample is determined as a key sample.

[0093] Specifically, the embodiment provides an implementation of screening key samples affected by the confounding effect. The prompts with different information contents (P1 simple prompt and P2 complex prompt) are compared to screen samples with significant output differences.

[0094] It can be understood that the defined simple and complex prompts are different for different prediction tasks.

[0095] In possible embodiments, as in the entity recognition task, P1 only provides the task name and a simple description (depending on the pre-training knowledge of the model), P2 provides a detailed task description and solution details (fully activates the model's ability), and if the outputs of the same sample under P1 and P2 are inconsistent, it is determined that the sample is a key sample (pre-training knowledge interference leads to unstable prediction).

[0096] Specifically, in the entity recognition task of the present embodiment, if a given input sample s and an entity type candidate set l are given, the prompts for the entity recognition task are as follows:

[0097] Prompt 1: Do an entity recognition task to identify words with specific meanings (entities) in the text. Mainly including names, place names, organization names, proper nouns, etc. Label the words that need to be identified in the text sequence. Select the most likely result from the label set and output. The input sample is s, and the candidate label set is l.

[0098] Prompt 2: Do an entity recognition task to identify and label words with specific meanings (entities) in the text one by one. Entities are a class of words with specific meanings, and are specific things that exist objectively. Entities mainly include names, place names, organization names, proper nouns, etc. Label the words that need to be identified in the text sequence. In identifying entities, special cases such as one word with multiple meanings, nested entities, or entities with intervals need to be paid special attention. Given the candidate set of labels l, select the most likely entity type result from the label set for each selected word. The input sample is s, and the candidate label set is l.

[0099] In detail, when positioning key data samples, two different prompts are designed to guide the output of the large model. The first prompt P1 only contains less task information, such as the task name and a simple task description. The purpose of P1 is to explore the pre-training knowledge of the large model by activating it with simple prompt information, and to use its memory knowledge to understand and solve the task. The second prompt P2 includes a detailed description of the current task and details that need to be paid attention to when solving the task. The purpose of P2 is to explore the pre-training knowledge of the large model by providing more rich task information and more fully activating and prompting the pre-training knowledge of the large model, so as to better solve the current task. Taking the entity recognition task as an example:

[0100] P1 and P2 will get two outputs for the same sample, and through the prompts with different information contents in all sample data, the samples different from the simple prompt and the complex prompt are found, and the samples are taken as the samples far away from the semantic distance of the large model, and the perturbation enhancement can be performed at the end.

[0101] For the above key samples, the application generates corresponding adversarial samples. Different from the picture adversarial sample generation method, for the text field, since the input text is discrete data, the result obtained by directly adding interference to the whole text usually has no meaning. Therefore, in the field of text adversarial sample generation, usually first find the position of the word (phrase) that has the greatest impact on the model in the input sample, and then perform character-level, word-level interference on the text data at this position, such as using spelling errors, homophones, synonyms and other strategies, so as to generate adversarial samples. Therefore, how to find the key position that has the greatest impact on the model is a key problem of text adversarial sample generation. For this problem, the project analyzes it in two scenarios of black box and white box.

[0102] In some possible embodiments of the application, the black box method comprises:

[0103] Traverse all positions of the input sample, replace the i-th position word with a mask to form a new input sample ;

[0104] Determine the importance score of each position of the input sample through the model output:

[0105] (1),

[0106] Wherein, is the position importance score of the i-th position of the input sample, is the encoding vector of the i-th position of the input sample, i=1~n, n is the total number of positions of the input sample, is the probability distribution before replacement, is the probability distribution after replacing the i-th position;

[0107] Select the input sample position with the highest score for character-level or semantic-level perturbation.

[0108] Specifically, the embodiment provides an implementation manner of the black box method, traverses the positions of the input sample, replaces [MASK] and observes the output probability change, calculates the position importance score, and selects the position with the highest score for perturbation (such as homophone replacement).

[0109] Specifically, in a black-box scenario, only the output and its corresponding probability are typically available. Therefore, this embodiment identifies the key positions that have the greatest impact on the model by analyzing the degree of change in these probabilities. Specifically, it iterates through all positions of the input sample, replacing the word at the i-th position with "MASK," thus forming a new input. The importance score for each position is obtained by analyzing the changes in the model output p. The importance score for the i-th position is... The calculation method is as follows:

[0110] ,

[0111] Based on the importance score obtained Select key locations:

[0112] (2),

[0113] in, , y represents the degree of influence of the i-th position on the model output, and y is the corresponding label.

[0114] Different levels of perturbation are applied to position j to generate adversarial examples.

[0115] In some possible embodiments of the present invention, the white-box method includes:

[0116] The optimal perturbation vector is generated based on gradient-based adversarial example generation theory. ;

[0117] Iterate through all positions of the input sample, replacing the word at the i-th position with a mask to form a new input sample. ;

[0118] Will The encoded vector and the optimal perturbation vector By comparing the input sample locations, the location with the smallest distance is selected as the optimal perturbation location.

[0119] Specifically, this embodiment provides an implementation of a white-box method to generate the optimal perturbation vector. (Based on gradient optimization), compare [MASK] positional encoding with Based on the similarity, select the position with the largest perturbation amplitude.

[0120] Specifically, in a black-box scenario, this embodiment finds the optimal perturbation location by analyzing the model loss and its gradient changes. Specifically, firstly, the optimal perturbation vector is generated based on gradient-based adversarial example generation theory. Secondly, iterate through all positions of the input sample, replacing the word at the i-th position with MASK, thus forming a new input. Finally, the obtained vector after encoding is compared with the optimal perturbation vector, and the position with the minimum distance is selected as the optimal perturbation position. The obtained vector after encoding is compared with the optimal perturbation vector, and the position with the minimum distance is selected as the optimal perturbation position.

[0121] (3).

[0122] The adversarial sample obtained based on the above method has the maximum perturbation amplitude on the model, and can more effectively evaluate the robustness of the large model.

[0123] In some possible embodiments of the present application, the adversarial sample includes the following types:

[0124] An adversarial prompt, which injects interference in the task instruction;

[0125] An adversarial content, which modifies key words in the input sample.

[0126] Specifically, the embodiment provides an implementation of an adversarial sample, which can include an adversarial prompt and an adversarial content. The adversarial prompt refers to injecting interference in the task instruction, and the adversarial content refers to modifying key words in the input sample. The interference granularity includes character level, word level, sentence level, etc.

[0127] In possible embodiments, as shown in Figure 3 The input of the large model is divided into two parts, a prompt and main content, wherein the prompt part is used to tell the model the task type or the required answer type, and the main content part refers to the text content that the model is expected to process. According to the interference position, the robustness test data set can be divided into three parts: an adversarial prompt, an adversarial content, and an adversarial input, wherein the adversarial prompt is to generate an adversarial prompt for the prompt, and the main content is still clean main content; the adversarial content is to generate an adversarial main content for the main content, and the prompt is still clean prompt; the adversarial input is to generate an adversarial prompt and an adversarial main content for the prompt and the main content. According to the perturbation granularity, the interference can be divided into character level, word level, sentence level, and mixed level. The interference tools include TextBugger, TextFooler, CheckList, and artificial synthesis, etc. For example, see Figure 3In the example part, in the input part of the clean sample, the clean prompt and the clean main content are "Translate the following sentence into English" and "I am very happy today" respectively, and the character level, word level, sentence level or mixed level interference is performed on the clean sample to generate the adversarial prompt, the adversarial content and the adversarial input. Among them, the adversarial prompt includes the adversarial prompt "Translate the following sentence into English <interference content>" and the clean main content "I am very happy today", the adversarial content includes the clean prompt "Translate the following sentence into English" and the adversarial main content "I am very happy today <interference content>", and the adversarial input includes the adversarial prompt "Translate the following sentence into English <interference content>" and the adversarial main content "I am very happy today <interference content>".

[0128] In possible embodiments, the generated adversarial samples are used to evaluate the robustness of the model, to test the stability and anti-interference ability of the model under input disturbance.

[0129] Specifically, the character level disturbance (such as misspelling, spelling error) is used to test the sensitivity of the model to subtle text changes; the semantic level disturbance (such as synonym replacement, sentence adjustment) is used to evaluate the understanding ability of the model to semantic consistency; the adversarial prompt (such as misleading task instruction) is used to verify the resistance of the model to instruction disturbance. The quantitative indicators include the accuracy drop amplitude: the improvement degree of the model prediction error rate after disturbance, and the output consistency: the fluctuation range of the results of the same problem under different disturbances.

[0130] In possible embodiments, the generated adversarial samples are used to expose potential vulnerabilities of the model, and the weak links of the model are revealed through targeted attacks.

[0131] Specifically, such as knowledge bias vulnerability: against the pre-training knowledge mixing effect (such as backdoor path disturbance), generate misleading samples, verify whether the model over-reliance on prior knowledge. Semantic understanding vulnerability: test the model's ability to analyze complex semantics such as polysemy and entity nesting. Provide specific direction for model optimization (such as adjusting the distribution of pre-training data, enhancing the generalization ability of specific tasks).

[0132] In possible embodiments, the generated adversarial samples are used to verify the effectiveness of causal analysis, and the conclusions of the structural causal model (SCM) are tested through adversarial samples.

[0133] Specifically, first, the causal model is located, and the samples susceptible to confounding factors (such as pre-training knowledge K) are identified through SCM. Adversarial sample generation, specifically generate disturbance for these samples, and observe whether the model fails as predicted by the causal model. Verify the accuracy of causal inference, form a "theoretical analysis → practical verification" closed loop.

[0134] The embodiments of the present application promote the evolution of the model to be more robust, more reliable and more secure by generating and applying the adversarial samples.

[0135] In some possible embodiments of the present application, the indicators for evaluating the large model by using the adversarial samples include one or more of the following:

[0136] The accuracy drop, the attack success rate, the semantic consistency, the robustness, the prompt sensitivity, the semantic understanding and generalization ability, the defense ability and the causal explainability.

[0137] Specifically, the embodiments of the present application provide an indicator for evaluating the large model, and the multi-dimensional evaluation indicators collectively constitute a comprehensive evaluation system for the intelligent level of the large model, overcome the pre-training knowledge bias, improve the stability and reliability of the model, and provide technical support for the research and development of large models in key fields.

[0138] Specifically, for the evaluation of robustness, the core evaluation target is to evaluate the stability and anti-interference ability of the model when facing input disturbance or adversarial attack. The adversarial sample test is to generate character-level, word-level or sentence-level adversarial samples (such as misspelling, synonym replacement), and detect whether the model prediction result changes significantly. The key indicators include disturbance amplitude tolerance: the consistency of the model output after disturbance (such as accuracy drop), backdoor path interference resistance: the resistance effect of the model to the pre-training knowledge mixing effect (analyzed by a causal model).

[0139] For prompt sensitivity, the evaluation target is to measure the response difference of the model to different information content prompts, and reflect the degree of its dependence on pre-training knowledge. The test method is to compare the output results of simple prompts (P1) and complex prompts (P2). The key indicators include output consistency, that is, the consistency of the prediction results of the same sample under P1 and P2 (the greater the difference, the higher the prompt sensitivity), and the degree of knowledge bias, that is, the adaptability of the model to complex task description (such as whether it can overcome the misleading of pre-training knowledge).

[0140] For the evaluation of semantic understanding and generalization ability, the evaluation target is to verify the depth of the model in understanding complex semantics, polysemy, entity nesting and other scenes. The test method includes key sample test, which observes whether the model can accurately identify the samples with distant semantic distance (such as polysemy, rare entities), and evaluates the model performance under unseen tasks or data distribution.

[0141] For defense capability evaluation, the evaluation target is to analyze the anti-interference capability of the analysis model under different attack scenarios. The black box scenario generates adversarial samples based only on the output probability change, tests the defense of the model against unknown attacks. The white box scenario uses model gradient to generate optimal perturbation, and evaluates the resistance of the model to internal mechanism exposure risk. The key indicators include attack success rate: the probability of successfully misleading the model by adversarial samples, and perturbation detection capability: whether the model can identify and filter adversarial inputs.

[0142] For causal interpretability evaluation, the evaluation target is to analyze the causal logic of the model prediction result through a structural causal model (SCM). The test methods include confounding effect positioning: identifying the interference path of pre-training knowledge (K) on the prediction result (Y) (such as the backdoor path X←E←K→V→Y), and causal intervention effect: verifying whether the model can weaken the influence of confounding factors by adjusting the answer generation constraint (V).

[0143] In a specific embodiment, the method for evaluating large models based on adversarial attacks provided by the present scheme comprises the following steps:

[0144] Step one: build an SCM to analyze the interference path of pre-training knowledge on entity type labeling;

[0145] Step two: input the sample "Beijing University was founded in 1898", and use P1 (simple prompt) and P2 (complex prompt) to obtain the output respectively;

[0146] P1 output: label "Beijing University" as "institution name";

[0147] P2 output: label "1898" as "time", and determine it as a key sample (P1 does not activate the time entity recognition capability);

[0148] Step three: generate an adversarial sample "1898mou" at the "1898" position, and test whether the model can still correctly identify it as "time";

[0149] Step four: calculate the accuracy drop (from 95% to 70%), and verify the insufficient robustness of the model.

[0150] The method for evaluating large models based on adversarial attacks provided by the present scheme filters key samples through a causal model, reduces the consumption of full-scale test resources, improves the evaluation efficiency, exposes model knowledge bias and semantic understanding defects by combining backdoor path analysis and adversarial attacks, and accurately locates vulnerabilities; supports black box (commercial model evaluation) and white box (internal tuning) dual scenarios, and adapts to high security requirements

[0151] In some specific embodiments of the present application, as shown in Figure 4 The present scheme provides a large model evaluation device based on adversarial attacks, which comprises:

[0152] The confounding effect analysis module 41 is used to analyze the confounding effects of confounding factors on the prediction results of the large model through confounding paths using a pre-built structural causal model.

[0153] The key sample screening module 42 is used to screen key samples affected by confounding effects by comparing the output differences of different prompts based on the analysis results of the structured causal model.

[0154] The adversarial sample generation module 43 is used to generate adversarial samples for the key samples using a black-box method or a white-box method.

[0155] Evaluation module 44 is used to evaluate the large model using the adversarial examples.

[0156] The large model evaluation device based on adversarial attacks provided in this embodiment of the invention has a similar implementation principle and beneficial effects to the large model evaluation method based on adversarial attacks shown in the above embodiments. Please refer to the implementation principle and beneficial effects of the large model evaluation method based on adversarial attacks shown in the above embodiments, which will not be repeated here.

[0157] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a large model evaluation method based on adversarial attacks. This method includes: using a pre-built structural causal model to analyze the confounding effects of confounding factors on the prediction results of the large model through confounding paths; based on the analysis results of the structural causal model, screening key samples affected by confounding effects by comparing the output differences of different prompts; generating adversarial samples for the key samples using black-box or white-box methods; and evaluating the large model using the adversarial samples.

[0158] In addition, the logic instructions in the memory 530 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0159] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the large model evaluation method based on the adversarial attack provided by the above-mentioned methods, which comprises: analyzing the confounding effect of confounding factors on the prediction result of the large model through confounding paths by using a pre-constructed structural causal model; based on the analysis result of the structural causal model, screening key samples affected by the confounding effect by comparing the output difference of different prompts; generating adversarial samples for the key samples by using a black box method or a white box method; and evaluating the large model by using the adversarial samples.

[0160] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the large model evaluation method based on the adversarial attack provided by the above-mentioned methods, which comprises: analyzing the confounding effect of confounding factors on the prediction result of the large model through confounding paths by using a pre-constructed structural causal model; based on the analysis result of the structural causal model, screening key samples affected by the confounding effect by comparing the output difference of different prompts; generating adversarial samples for the key samples by using a black box method or a white box method; and evaluating the large model by using the adversarial samples.

[0161] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0162] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0163] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A large model evaluation method based on adversarial attack, characterized in that, The large model is an entity recognition task for discrete data input text, which identifies words with specific meanings in the text, and the method comprises: Using a pre-constructed structural causal model to analyze the confounding effects of confounding factors on the prediction results of the large model through confounding paths, the structural causal model is specifically used to identify the confounding effects of pre-training knowledge on the model output; Based on the analysis result of the structural causal model, by comparing the output difference of different prompts, the key samples affected by the confounding effect are screened out, which are different results obtained by different prompts for the same input text; the different prompts include simple prompts and complex prompts, wherein the simple prompts only contain task names and general expressions, and the complex prompts include one or more of detailed task descriptions, solution steps, potential interference items, constraint conditions and examples; Accordingly, based on the analysis result of the structural causal model, by comparing the output difference of different prompts, the key samples affected by the confounding effect are screened out, specifically including: Splicing the same input sample with simple prompts and complex prompts respectively to form two input sequences; Input the two input sequences into the large model to generate prediction results, and record the probability distribution of each prediction result; Calculate the confidence difference of the two prediction results, if the calculation result of the confidence difference is greater than the preset threshold, the input sample is determined as a key sample; For the key sample, an adversarial sample is generated by a black box method or a white box method, the white box method comprising: Generating an optimal perturbation vector based on the gradient-based adversarial sample generation theory; Traverse all positions of the input sample, replace the i-th position word with a mask to form a new input sample; Compare the encoded vector with the optimal perturbation vector, and select the input sample position with the smallest distance as the optimal perturbation position; The black box method comprises: Traverse all positions of the input sample, replace the i-th position word with a mask to form a new input sample; Determine the importance score of each position of the input sample through the model output; Select the word of the input sample position with the highest score for character-level, word-level, sentence-level and mixed-level perturbation; Using the adversarial sample, the large model is evaluated, and the evaluation indexes include semantic understanding and generalization ability evaluation, and causal interpretability.

2. The large model evaluation method based on adversarial attack according to claim 1, wherein, The construction process of the structural causal model comprises: Defining the variables of the large model and the causal relationship between the variables; Based on the causal relationship, a causal graph DAG is drawn; Establish a structural equation for each variable to mathematically describe the causal relationship between the variables.

3. The large model evaluation method based on adversarial attack according to claim 2, characterized in that, Analyzing the confounding effects of confounding factors on the prediction results of the large model through confounding paths, specifically including: Determine the confounding factors and the confounding paths corresponding to the confounding factors through the causal graph DAG; Block the confounding paths by eliminating bias through control variables or intervention; Through the structural equation, calculate the effect estimation difference before and after blocking the confounding paths, and determine the calculation result of the effect estimation difference as the confounding effect of each confounding factor on the prediction results of the large model through the confounding paths.

4. The large model evaluation method based on adversarial attack according to claim 1, wherein, The importance score of each position of the input sample is determined through the model output, and the specific formula is: ， wherein, is a position importance score of the i-th position of the input sample, is an encoding vector of the i-th position in the input sample, i = 1 ~ n, n is the total number of positions of the input sample, is a probability distribution before replacement, is a probability distribution after replacing the i-th position.

5. The large model evaluation method based on adversarial attack according to claim 1, wherein, The adversarial sample includes the following types: Adversarial prompts, injecting interference in task instructions; Adversarial content, modifying key vocabulary in input samples.

6. The large model evaluation method based on adversarial attack according to claim 1, wherein, The indicators for evaluating the large model using the adversarial sample include one or more of the following: Accuracy drop, attack success rate, semantic consistency, robustness, prompt sensitivity, semantic understanding and generalization ability, and defense ability.

7. A large model evaluation device based on adversarial attacks, characterized in that, The large model is an entity recognition task for discrete data input text, which identifies words with specific meanings in the text, and the device includes: A confounding effect analysis module for analyzing the confounding effect of confounding factors on the prediction results of the large model through confounding paths using a pre-constructed structural causal model, which is specifically used to identify the confounding effect of pre-training knowledge on model output; A key sample screening module for screening key samples that have different results for the same input text under different prompts based on the analysis results of the structural causal model by comparing the output differences of different prompts; the different prompts include simple prompts and complex prompts, wherein the simple prompts only include task names and general expressions, and the complex prompts include one or more of detailed task descriptions, solution steps, potential interference items, constraints, and examples; Accordingly, based on the analysis results of the structural causal model, the key samples affected by the confounding effect are screened by comparing the output differences of different prompts, specifically including: Splicing the same input sample with simple prompts and complex prompts to form two input sequences; Inputting the two input sequences into the large model to generate prediction results and recording the probability distribution of each prediction result; Calculate the confidence difference of the two prediction results, if the calculation result of the confidence difference is greater than the preset threshold, the input sample is determined as a key sample; An adversarial sample generation module for generating adversarial samples for the key samples through a black box method or a white box method, wherein the white box method includes: Gradient-based adversarial sample generation theory generates optimal perturbation vector ; traversing all positions of the input sample, replacing the i-th position word by the mask, forming a new input sample ; Will The encoded vector and the optimal perturbation vector By comparing the input sample locations, the location with the smallest distance is selected as the optimal perturbation location; The black box method includes: traversing all positions of the input sample, replacing the i-th position word by the mask, forming a new input sample ; Determine the importance score of each position of the input sample through the model output; Select the word at the highest scoring input sample position for character-level, word-level, sentence-level, and mixed-level perturbation; An evaluation module for evaluating the large model using the adversarial sample, and the evaluation indicators include semantic understanding and generalization ability evaluation and causal interpretability.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to realize the large model evaluation method based on adversarial attack according to any one of claims 1 to 6.

Images