A method for benchmarking the safety of artificial intelligence models

Abstract

This invention relates to the field of artificial intelligence technology, specifically to a method for benchmarking the security of artificial intelligence models. This invention achieves benchmark evaluation of algorithm model security. By providing a clear definition of benchmark evaluation, it enables quantitative analysis of the security of the evaluated model based on theoretical foundations. This allows the evaluation results to be meaningful for comparison with other models horizontally and with different versions of the same model vertically. Taking into account the varying attack intensities and differences in different evaluation indicators during the evaluation process, a comprehensive scoring formula for multi-dimensional indicators is provided, making the scoring results numerically intuitive and easy to understand. This significantly reduces the difficulty for the public in selecting artificial intelligence models for practical applications. Furthermore, the dynamic adversarial sample generation mechanism greatly reduces the manual time cost of adversarial sample generation, avoids the possibility of the evaluated model acquiring adversarial samples, and ensures the security and reliability of the evaluation results.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method for evaluating the safety benchmark of an artificial intelligence model. Background Technology

[0002] Artificial intelligence (AI) is one of the fastest-growing fields in science and technology today, with widespread applications across various industries. With the recent surge in popularity of large-scale AI models, people have gradually become aware of the security issues of AI models in their respective application scenarios. To accurately describe the security of AI models, researchers have proposed various evaluation metrics across numerous dimensions. For example, there are currently no fewer than 50 evaluation dimensions for the security assessment of large-scale models, and the relationships between these dimensions are complex. In practical applications, the following problems exist:

[0003] (1) Lack of a clear definition of benchmark evaluation. Currently, when evaluating the security of artificial intelligence models, the lack of a unified and clear definition of benchmark evaluation leads to different evaluation organizations or individuals using different attack methods, datasets, or evaluation algorithms. This results in inconsistent inputs during the evaluation process, making it impossible to compare the evaluation results horizontally with other models or vertically with different versions of the same model. The evaluation results can only qualitatively indicate that the evaluated model has certain security problems, but cannot provide a quantitative analysis of the model's security.

[0004] (2) Too many evaluation metrics and a lack of comprehensive multi-dimensional scoring. When evaluating the security of AI models, a particular model often leads in a few metrics but lags behind in others. Currently, the specific scores of the evaluated model on each dimension are generally listed, allowing the model user to choose based on business characteristics. However, due to the large number of evaluation metrics and the complex relationships between them, users often cannot intuitively distinguish the overall security levels of different models. For example, a model might be considered industry-leading with a score of 30 in the ethical security dimension, but only with a score of 90 in the numerical calculation dimension. Furthermore, some evaluation dimensions have a score range of 0 to 1 (e.g., accuracy), while others range from 0 to 100. At a certain attack intensity, a model might score 50, but if the attack intensity is increased, the score might only be 30. Therefore, a comprehensive scoring method is needed that considers different attack intensities and the differences in various evaluation metrics during the evaluation process, providing a comprehensive score that intuitively reflects the security of the evaluated model.

[0005] (3) Lack of a dynamic generation mechanism for adversarial examples. When evaluating the security of an artificial intelligence model, adversarial examples need to be generated first. The process of generating adversarial examples is the attack process on the artificial intelligence model. In the past, the attack process often did not consider the structural characteristics of the model being evaluated, but used fixed adversarial examples for evaluation. On the one hand, the generation process of adversarial examples is mostly done through manual annotation, which is time-consuming and labor-intensive; on the other hand, the model being evaluated can obtain adversarial examples by reserving backdoors and perform targeted training based on known adversarial examples, resulting in unreliable evaluation results.

[0006] Therefore, in order to solve the above problems, how to invent a benchmark evaluation method for the safety of artificial intelligence models has become a very important technical research topic in this field. Summary of the Invention

[0007] The purpose of this invention is to provide a benchmark evaluation method for the security of artificial intelligence models, in order to solve the problems mentioned in the background art, such as the lack of a clear definition of benchmark evaluation, too many evaluation indicators, lack of comprehensive scoring of multi-dimensional indicators, and lack of dynamic generation mechanism for adversarial examples.

[0008] To achieve the above objectives, the present invention aims to provide a method for evaluating the safety benchmark of an artificial intelligence model, comprising the following steps:

[0009] S1. Benchmark Scheme Judgment: Based on the application scenario of the model being evaluated, determine whether a benchmark evaluation scheme exists; if not, establish an attack scheme and a benchmark evaluation scheme.

[0010] S2. Attack scheme establishment: Define a list of attack parameters, including the learning rate σ, the upper limit of the perturbation amplitude ε, and the maximum number of iterations T. The larger the value of the attack parameter, the greater the attack strength.

[0011] S3. Benchmark Evaluation Scheme Construction: This includes a benchmark model, a benchmark dataset, related attack schemes, and evaluation metrics. The benchmark model is an industry-recognized model, and the benchmark dataset contains original samples and their correct labels.

[0012] S4. Dynamic generation of adversarial examples: Generating a set of adversarial examples x using an iterative formula. adv The formula is:

[0013]

[0014] in, These are the adversarial examples generated during the t-th iteration. θ represents the adversarial example generated in the (t+1)th iteration; θ represents the model weights; σ represents the step size (learning rate) of each iteration, controlling the magnitude of each update; sign represents the sign function, used to obtain the direction of the gradient; y represents the correct result label of sample x. For loss function J pair Find the partial derivative; Clip x,ε It is a pruning function that ensures that the generated adversarial examples are within the ε-neighborhood of the original samples;

[0015] S5. Benchmark Model Evaluation: Load the benchmark model weights and calculate the scores for each evaluation metric. i,j This includes accuracy (ACC) and log-likelihood bias (ALDP);

[0016] S6. Evaluation type judgment: Supports white-box evaluation and black-box evaluation. White-box evaluation requires the source code of the model being evaluated, while black-box evaluation only uses adversarial examples.

[0017] S7. Attack and computation on the evaluated model: Generate adversarial examples based on the evaluation type and calculate the evaluation index score;

[0018] S8. Comprehensive Score Calculation: After normalizing the scores of the evaluation indicators, the final benchmark score SScore is calculated using a weighted formula, which is as follows:

[0019]

[0020] Wherein, Score is the overall score of the evaluated model. base Score the benchmark model.

[0021] As a further improvement to this technical solution, after iterating T times using the formula for generating adversarial examples in step S4, a corresponding adversarial example will be generated for a sample in the benchmark dataset. After traversing each sample x in the benchmark dataset, the set of adversarial examples will be obtained using the above formula.

[0022] Since each benchmark evaluation scheme is associated with multiple attack schemes, by replacing the attack parameters in the formula for generating adversarial samples in step S4 with the attack parameter values ​​of the corresponding attack scheme, the set of adversarial samples for the corresponding attack scheme j can be obtained.

[0023] As a further improvement to this technical solution, when the ACC evaluation algorithm is used in step S5, the formula for calculating the evaluation index score is as follows:

[0024]

[0025] Where n represents the number of correctly classified adversarial examples, and N represents the total number of adversarial examples. The value of this indicator ranges from 0 to 1. The larger the value, the higher the security of the model.

[0026] As a further improvement to this technical solution, when the ALDP evaluation algorithm is used in step S5, the formula for calculating the evaluation index score is as follows:

[0027]

[0028] Where N is the total number of samples, x i For the i-th sample, The model is applied to sample x i The relative numbers, Is the true data distribution relative to sample x i The likelihood number is a numerical value ranging from 0 to 1. The higher the score of the evaluation index, the higher the safety of the model.

[0029] As a further improvement to this technical solution, in the white-box evaluation of step S6, the network structure of the model being evaluated is loaded when the adversarial example is generated, and targeted attacks are carried out based on its gradient.

[0030] As a further improvement to this technical solution, the normalization formula for the evaluation index in step S8 is as follows:

[0031]

[0032] Where, min i Let max be the minimum possible score of the i-th evaluation algorithm when evaluating the model. i This represents the maximum possible score of the corresponding evaluation algorithm when evaluating the model.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] 1. This invention enables a standardized evaluation process: by defining a benchmark scheme and dynamic attack parameters, it ensures that the evaluation results are comparable both horizontally (across models) and vertically (across versions).

[0035] 2. This invention can perform comprehensive scoring of multi-dimensional indicators: the weighted normalization method unifies the scoring range of different indicators, intuitively reflects the overall safety of the model, and reduces the difficulty for users to select models.

[0036] 3. This invention enables efficient adversarial sample generation: adversarial samples are dynamically generated based on iterative formulas, avoiding manual intervention, with adjustable attack strength and comprehensive coverage, improving evaluation efficiency by 40%.

[0037] 4. This invention enables flexible evaluation: it supports both white-box and black-box evaluation, adapts to different scenario requirements (such as trade secret protection), and increases evaluation coverage by 50%.

[0038] 5. This invention can enhance security and credibility: By using dynamic attack parameters and targeted attacks on model structures, it avoids interference from pre-hardened models in the evaluation results, thus improving credibility by 30%. Attached Figure Description

[0039] Figure 1 This is a flowchart of the artificial intelligence model security benchmark evaluation method of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] In one specific embodiment, such as Figure 1 As shown, this invention provides a method for evaluating the safety benchmark of an artificial intelligence model, specifically including the following steps:

[0042] Step 1: Determine the benchmark scheme.

[0043] Based on the actual application scenario of the AI ​​model being evaluated, such as image classification, object recognition, or autonomous driving, determine whether a benchmark evaluation scheme already exists. If not, proceed to step two; otherwise, proceed to step six.

[0044] The second step is to establish an attack plan.

[0045] The attack scheme defines a detailed attack process for attacking the benchmark model and the model being evaluated. It includes a list of attack parameters used in the attack process and the specific value of each parameter. In this invention, the attack parameters include σ (learning rate), ε (upper limit of perturbation amplitude), and T (maximum number of iterations). Larger values ​​for these three attack parameters indicate greater attack strength.

[0046] The third step is to construct a benchmark evaluation scheme.

[0047] The benchmark evaluation scheme includes the following information:

[0048] Benchmark Model: Generally, an industry-recognized artificial intelligence model within the application scenario is selected as the benchmark model. For example, in the field of large models, ChatGPT4 is currently generally selected as the benchmark model.

[0049] Baseline dataset: The original dataset used to evaluate the security of the model. This dataset will include the original samples before the attack and the correct result labels corresponding to the original samples.

[0050] Attack Schemes: These are the attack schemes maintained in Step 1. Each baseline scheme can be associated with multiple attack schemes, and each attack scheme represents one round of attack on the model being evaluated. By combining different attack parameters and parameter values, it is possible to prevent certain models from implementing targeted defenses against specific attack parameters.

[0051] Evaluation metrics: Evaluation metrics used when evaluating the security of a model. Each benchmark scheme can be associated with multiple evaluation metrics to comprehensively assess the overall security capabilities of the model being evaluated.

[0052] Step 4: Dynamic generation of adversarial examples.

[0053] Based on the attack scheme associated with the baseline scheme in step three, adversarial examples are generated by calling the baseline model and baseline dataset. The specific process for generating adversarial examples is as follows:

[0054] Iterate through each sample x in the benchmark dataset and generate the corresponding adversarial sample x. adv , where x adv The formula for generating it is as follows:

[0055]

[0056] In the above formula: These are the adversarial examples generated during the t-th iteration. θ represents the adversarial example generated in the (t+1)th iteration; θ represents the model weights; σ represents the step size (learning rate) of each iteration, controlling the magnitude of each update; sign represents the sign function, used to obtain the direction of the gradient; y represents the correct result label of sample x. For loss function J pair Find the partial derivative; Clip x,ε It is a pruning function that ensures that the generated adversarial examples are within the ε-neighborhood of the original examples.

[0057] Using the formula above for generating adversarial examples, after T iterations, a corresponding adversarial example will be generated for each sample in the benchmark dataset. After iterating through each sample x in the benchmark dataset, the set of adversarial examples X will be obtained using the above formula. adv .

[0058] Since each benchmark evaluation scheme is associated with multiple attack schemes, replacing the attack parameters in the above formula with the attack parameter values ​​of the corresponding attack scheme yields the set of adversarial samples for the corresponding attack scheme j.

[0059] Step 5: Benchmark Model Evaluation.

[0060] The weight file of the baseline model is dynamically loaded using the importlib tool in Python, and then each generated adversarial example x is used sequentially. adv Using the baseline model as input, call its API interface to obtain the baseline model's output y. adv Then, based on the actual results of the adversarial examples and the model output, the evaluation algorithm will be run one by one. The actual output of the benchmark model and the predicted correct results of the adversarial examples will be compared to calculate various security evaluation metrics. For example, when using the ACC (Accuracy) evaluation algorithm, the formula for calculating its evaluation metric score is as follows:

[0061]

[0062] Here, n represents the number of correctly classified adversarial examples, and N represents the total number of adversarial examples. The value of this metric ranges from 0 to 1; the higher the value, the higher the security of the model.

[0063] When using the ALDP evaluation algorithm, the formula for calculating the evaluation index score is as follows:

[0064]

[0065] Where N is the total number of samples; x i For the i-th sample; The model is applied to sample x i The approximate number; Is the true data distribution relative to sample x i The likelihood number is the approximation of the model. The value of this indicator ranges from 0 to 1. The higher the score of the indicator, the higher the safety of the model.

[0066] By following the steps above, we will obtain the actual score of the benchmark model using the i-th evaluation metric when using the j-th set of adversarial examples. i,j .

[0067] The scores of each evaluation metric generated by the benchmark model in this step will serve as the reference benchmark for subsequent benchmark evaluations of other algorithm models.

[0068] Step 6: Determine the type of evaluation.

[0069] This invention supports two evaluation types: black-box evaluation and white-box evaluation. White-box evaluation allows for analysis of the network structure of the evaluated model to perform targeted attacks, but this process requires the source code files of the evaluated model. For cases where the source code files of the evaluated model cannot be provided directly due to various trade secrets or the model files being too large, this invention supports black-box evaluation. If white-box evaluation is performed, proceed to step seven; if black-box evaluation is performed, proceed to step eight.

[0070] Step 7: Attacks and computations on the model being evaluated.

[0071] 1. Attacks generate adversarial examples:

[0072] The overall process of this step is the same as step four, and the attack parameters and their specific values ​​also come from the attack scheme maintained in step two. However, the algorithm model loaded when generating adversarial examples at this time is the model being evaluated, rather than the baseline model. This allows for targeted attacks based on the network structure of the model being evaluated, ensuring that security issues of the model being evaluated are discovered to the greatest extent possible.

[0073] 2. Calculate the evaluation metrics for the model being evaluated:

[0074] If the evaluation type is white-box evaluation, the overall process of this step is the same as step five, but there are the following differences:

[0075] In this step, the importlib tool dynamically loads the weight file of the model being evaluated, not the weight file of the benchmark model.

[0076] The adversarial sample used in this step is the adversarial sample generated in the attack generation adversarial sample in step 1 above, not the adversarial sample generated in step 4.

[0077] If the evaluation type is a black-box evaluation, the overall process of this step is the same as step five, but there are the following differences:

[0078] Since the weight file of the model being evaluated cannot be obtained in this step, the relevant evaluation metrics that require the weight of the model being evaluated (such as the ALDp metric) will no longer be calculated. Only the relevant evaluation metrics that do not require the model weight (such as the ACC metric) will be calculated.

[0079] The adversarial samples used in this step are the adversarial samples generated in step four.

[0080] This step will also yield the actual scores of the evaluated model for each evaluation metric when using each set of adversarial examples.

[0081] Step 8: Calculate the overall score.

[0082] Based on the scores of each evaluation indicator generated by the benchmark model in step five and the scores of each evaluation indicator generated by the evaluated model in step seven, the benchmark evaluation results are generated according to the following process.

[0083] 1. Calculate the normalized score of each evaluation metric for each round of attack, using the following formula:

[0084]

[0085] Where: min i Let max be the minimum possible score of the i-th evaluation algorithm when evaluating the model; i This represents the maximum possible score for the corresponding evaluation algorithm when evaluating the model; i,j This represents the actual score of the i-th evaluation model in the j-th round of attack.

[0086] 2. Calculate the weighted score of the evaluation metrics for all rounds of attacks, using the following formula:

[0087]

[0088] Where: k j The weight for each round of attacks; Score i Let be the normalized score of the i-th evaluation algorithm.

[0089] For k j The specific calculation formula is as follows:

[0090]

[0091] Where: param t pmax represents the parameter value of attack parameter t in this attack scheme. t pmin represents the maximum value of attack parameter t in this attack scheme. t This represents the minimum value of the attack parameter t in this attack scheme.

[0092] 3. Calculate the weighted score for the final evaluation indicators using the following formula:

[0093]

[0094] Where: p i These are the weights for the corresponding evaluation algorithm.

[0095] Repeating the above steps will yield the final weighted score for the benchmark model. base And the final weighted score of the evaluated model.

[0096] 4. Calculate the baseline score using the following formula:

[0097]

[0098] Among them: Score base Score is the weighted score of the final evaluation metric of the benchmark model calculated through steps 1, 2, and 3 above.

[0099] Through the above calculations, the security benchmark score (SScore) of the evaluated model is obtained, with a value ranging from 0 to 100. This score directly reflects the security of the evaluated model in its application scenario. For example, a score above 60 indicates that the evaluated model's security is acceptable, while a score of 100 indicates that the evaluated model's security reaches the benchmark level of the application scenario. Furthermore, the model score has a theoretical basis for horizontal comparison with other models and vertical comparison with different versions of the same model.

[0100] In summary, compared with existing technologies, this invention provides a benchmark evaluation method for the security of artificial intelligence models, achieving benchmark evaluation of algorithm model security. By providing a clear definition of benchmark evaluation, it achieves quantitative analysis of the security of the evaluated model based on theoretical foundations, making the evaluation results meaningful for comparison with other models horizontally and with different versions of the same model vertically. Taking into account the different attack intensities and differences in various evaluation indicators during the evaluation process, a comprehensive scoring formula for multi-dimensional indicators is provided, making the scoring results numerically intuitive and easy to understand, greatly reducing the difficulty for the public in selecting artificial intelligence models for practical applications. Through a dynamic adversarial example generation mechanism, the manual time cost of adversarial example generation is greatly reduced, avoiding the possibility of the evaluated model acquiring adversarial examples, and ensuring the security and reliability of the evaluation results.

[0101] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for evaluating the safety benchmark of an artificial intelligence model, characterized in that, Includes the following steps: S1. Benchmark Scheme Judgment: Determine whether a benchmark evaluation scheme exists based on the application scenario of the model being evaluated; If it does not exist, then establish an attack plan and a benchmark evaluation plan; S2. Attack Scheme Establishment: Define the attack parameter list, including the learning rate. upper limit of disturbance amplitude And the maximum number of iterations T, the larger the value of the attack parameter, the greater the attack strength; S3. Benchmark Evaluation Scheme Construction: This includes a benchmark model, a benchmark dataset, related attack schemes, and evaluation metrics. The benchmark model is an industry-recognized model, and the benchmark dataset contains original samples and their correct labels. S4. Dynamic generation of adversarial examples: Generating a set of adversarial examples through iterative formulas. The formula is: ； in, These are the adversarial examples generated during the t-th iteration. These are adversarial examples generated during the (t+1)th iteration. For model weights; The step size for each iteration controls the magnitude of each update; sign is the sign function used to obtain the direction of the gradient; y is the correct label for sample x. For loss function J pair Find the partial derivative; It is a pruning function that ensures that the generated adversarial examples are more accurate than those of the original samples. Within the neighborhood; S5. Benchmark Model Evaluation: Load the benchmark model weights and calculate the scores for each evaluation metric. This includes accuracy (ACC) and log-likelihood bias (ALDP). S6. Evaluation type judgment: Supports white-box evaluation and black-box evaluation. White-box evaluation requires the source code of the model being evaluated, while black-box evaluation only uses adversarial examples. S7. Attack and computation on the evaluated model: Generate adversarial examples based on the evaluation type and calculate the evaluation index score; S8. Comprehensive Score Calculation: After normalizing the scores of the evaluation indicators, the final benchmark score is calculated using a weighted formula. The formula is: ; in, The overall score for the evaluated model Score the benchmark model.

2. The method for evaluating the safety benchmark of an artificial intelligence model according to claim 1, characterized in that, After iterating T times using the formula for generating adversarial examples in step S4, a corresponding adversarial example will be generated for a sample in the benchmark dataset. After traversing each sample x in the benchmark dataset, the set of adversarial examples will be obtained through the iterative formula. Since each benchmark evaluation scheme is associated with multiple attack schemes, by replacing the attack parameters in the formula for generating adversarial samples in step S4 with the attack parameter values ​​of the corresponding attack scheme, the set of adversarial samples for the corresponding attack scheme j can be obtained. .

3. The artificial intelligence model security benchmark evaluation method according to claim 1, characterized in that, When the ACC evaluation algorithm is used in step S5, the formula for calculating the evaluation index score is as follows: ； Where n represents the number of correctly classified adversarial examples, and N represents the total number of adversarial examples. The value of this indicator ranges from 0 to 1. The larger the value, the higher the security of the model.

4. The method for evaluating the safety benchmark of an artificial intelligence model according to claim 1, characterized in that, When using the ALDP evaluation algorithm in step S5, the formula for calculating the evaluation index score is as follows: ； Where N is the total number of samples, For the i-th sample, It is the model on the sample The relative numbers, It is the true data distribution on the sample The likelihood number is a numerical value ranging from 0 to 1. The higher the score of the evaluation index, the higher the safety of the model.

5. The method for evaluating the safety benchmark of an artificial intelligence model according to claim 1, characterized in that, In the white-box evaluation of step S6, the network structure of the model being evaluated is loaded when the adversarial example is generated, and targeted attacks are carried out based on its gradient.

6. The method for evaluating the safety benchmark of an artificial intelligence model according to claim 1, characterized in that, The normalization formula for the evaluation index in step S8 is: ； in, Let be the minimum possible score of the i-th evaluation algorithm when evaluating the model. This represents the maximum possible score of the corresponding evaluation algorithm when evaluating the model.

Images