Deep learning model evaluation method and device, electronic device and storage medium

Abstract

The invention relates to a deep learning model evaluation method and device, an electronic device and a storage medium, and the method comprises the steps: obtaining an original sample, and obtaining an initial confrontation sample for adding a disturbance term to the original sample; inputting the original sample into an ensemble learning model, processing the original sample by using each sub-model in the ensemble learning model to obtain a first output value corresponding to each sub-model, inputting the initial confrontation sample into the ensemble learning model, and processing the initial confrontation sample by using each sub-model to obtain a second output value corresponding to each sub-model; calculating a loss function result between the first output value and the second output value; based on a loss function result, optimizing a disturbance term, and generating a target confrontation sample; and attacking the to-be-evaluated deep learning model by using the target confrontation sample to generate an evaluation result for the deep learning model. Through the method and the device, the problem of high data dependence on an evaluation algorithm of a deep learning model is solved.

Description

Technical Field

[0001] The present application relates to the field of deep learning, and in particular to evaluation methods, devices, electronic devices and storage media for deep learning models. Background Art

[0002] Deep learning has been widely used in many fields and has significantly changed traditional research methods in many aspects, including autonomous driving, medical image segmentation, and identity recognition. However, the application of deep learning also faces a significant vulnerability, namely adversarial attacks. This attack causes the classifier to make incorrect predictions or not behave as expected by adding small perturbations to the input samples. Most adversarial attacks can only generate adversarial perturbations for specific images one by one based on the semantic content of the input samples, which is inefficient when evaluating the robustness of the target model trained on a large-scale dataset and has strong data dependence.

[0003] Currently, no effective solution has been proposed for the problem of strong data dependence of evaluation algorithms for deep learning models in related technologies. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device and storage medium for evaluating a deep learning model, so as to at least solve the problem in the related art that the evaluation algorithm of the deep learning model is highly dependent on data.

[0005] In a first aspect, an embodiment of the present application provides a method for evaluating a deep learning model, the method comprising:

[0006] Obtain an original sample, and obtain an initial adversarial sample by adding a perturbation term to the original sample;

[0007] Input the original sample into the ensemble learning model, use each sub-model in the ensemble learning model to process the original sample, obtain the first output value corresponding to each sub-model, and input the initial adversarial sample into the ensemble learning model, use each sub-model to process the initial adversarial sample, and obtain the second output value corresponding to each sub-model;

[0008] Calculating a loss function result between the first output value and the second output value; optimizing the perturbation term based on the loss function result to generate a target adversarial sample;

[0009] The target adversarial sample is used to attack the deep learning model to be evaluated, and an evaluation result for the deep learning model is generated.

[0010] In some embodiments, the calculating a loss function result between the first output value and the second output value includes:

[0011] Calculate the similarity result between the first output value and the second output value corresponding to each sub-model;

[0012] Obtaining a preset loss weight parameter assigned to each of the sub-models; and calculating the loss function result based on the loss weight parameter and the similarity result.

[0013] In some embodiments, the step of optimizing the perturbation term based on the loss function result and generating a target adversarial sample includes:

[0014] By changing the loss weight parameter, the loss function result is maximized to obtain an optimized loss weight parameter;

[0015] Iteratively calculate the optimized loss function result according to the optimized loss weight parameter;

[0016] Minimize the optimization loss function result by changing the disturbance term, and iteratively calculate the optimized disturbance term;

[0017] The optimized perturbation term is added to the original sample to obtain the target adversarial sample.

[0018] In some embodiments, maximizing the loss function result by changing the loss weight parameter to obtain an optimized loss weight parameter includes:

[0019] When the disturbance term is fixed, a first gradient of the loss function result with respect to the loss weight parameter is calculated, and the first gradient is reversely transmitted to the ensemble learning model, and a maximization iterative optimization is performed based on a preset first learning rate to obtain the optimized loss weight parameter;

[0020] The step of minimizing the optimization loss function result by changing the disturbance term and iteratively calculating the optimized disturbance term comprises:

[0021] When the optimization loss weight parameter is fixed, the second gradient of the optimization loss function result with respect to the disturbance term is calculated, and the second gradient is reversely transferred to the ensemble learning model, and minimization iterative optimization is performed based on a preset second learning rate to obtain the optimization disturbance term.

[0022] In some embodiments, using the target adversarial sample to attack a deep learning model to be evaluated and generating an evaluation result for the deep learning model includes:

[0023] Input the original sample into the deep learning model, output the original label, and input the target adversarial sample into the deep learning model, output the predicted label;

[0024] The original label and the predicted label are compared to generate the evaluation result.

[0025] In some embodiments, comparing the original label and the predicted label to generate the evaluation result includes:

[0026] Obtaining first sample identifiers corresponding to a plurality of the original samples and second sample identifiers corresponding to a plurality of the target adversarial samples;

[0027] Based on the first sample identifier and the second sample identifier, determining a correspondence between each of the original samples and each of the target adversarial samples;

[0028] Based on the corresponding relationship, the original label and the predicted label are compared to obtain the attack result of each target adversarial sample, and the attack results are summarized to generate the evaluation result.

[0029] In a second aspect, an embodiment of the present application provides an evaluation device for a deep learning model, comprising:

[0030] An acquisition module, used to acquire an original sample and acquire an initial adversarial sample by adding a perturbation term to the original sample;

[0031] An integrated learning model, used to input the original sample into the integrated learning model, use each sub-model in the integrated learning model to process the original sample, and obtain a first output value corresponding to each sub-model, and input the initial adversarial sample into the integrated learning model, use each sub-model to process the initial adversarial sample, and obtain a second output value corresponding to each sub-model;

[0032] An optimization module, configured to calculate a loss function result between the first output value and the second output value; based on the loss function result, optimize the disturbance term to generate a target adversarial sample;

[0033] An evaluation module is used to use the target adversarial sample to attack the deep learning model to be evaluated, and generate an evaluation result for the deep learning model.

[0034] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the evaluation method of the deep learning model as described in the first aspect above is implemented.

[0035] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the evaluation method of the deep learning model as described in the first aspect above.

[0036] Compared with the related art, the deep learning model evaluation method, device, electronic device and storage medium provided in the embodiments of the present application obtain the original sample and obtain the initial adversarial sample by adding a disturbance term to the original sample; input the original sample into the integrated learning model, use each sub-model in the integrated learning model to process the original sample, and obtain the first output value corresponding to each sub-model, and input the initial adversarial sample into the integrated learning model, use each sub-model to process the initial adversarial sample, and obtain the second output value corresponding to each sub-model; calculate the loss function result between the first output value and the second output value; based on the loss function result, optimize the disturbance term and generate a target adversarial sample; use the target adversarial sample to attack the deep learning model to be evaluated, and generate an evaluation result for the deep learning model.

[0037] Based on this, the perturbation terms of adversarial samples can be applicable to a variety of deep learning models, which improves the universality of adversarial samples. At the same time, the above-mentioned target adversarial samples are iteratively optimized and generated based on the output results of the integrated learning model, by calculating the loss function results between the first output value of the original sample and the second output value of the adversarial sample. There is no need to use the true label of the original sample data, thereby removing the dependence on the true label, making the target adversarial samples applicable to a variety of data-dependent scenarios, and there is no need to generate adversarial perturbations for specific images one by one based on the semantic content of the input sample, thereby effectively solving the problem of strong data dependence of the evaluation algorithm of the deep learning model and improving the evaluation efficiency of the deep learning model.

[0038] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 It is a hardware structure block diagram of a terminal of a deep learning model evaluation method according to an embodiment of the present application;

[0041] Figure 2 is a flow chart of a method for evaluating a deep learning model according to an embodiment of the present application;

[0042] Figure 3 is a flowchart of another deep learning model evaluation method according to an embodiment of the present application;

[0043] Figure 4 is a schematic diagram of a disturbance optimization method according to an embodiment of the present application;

[0044] Figure 5 It is a structural block diagram of an evaluation device for a deep learning model according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0046] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0047] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0048] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. Taking running on a terminal as an example, Figure 1 1 is a hardware structure block diagram of a terminal of a method for evaluating a deep learning model according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0049] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the evaluation method of the deep learning model in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0050] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0051] As described in the background technology, most adversarial attacks in the prior art can only generate adversarial perturbations for specific images one by one based on the semantic content of the input samples, resulting in insufficient efficiency in evaluating the robustness of deep learning models trained on large-scale data sets. Among them, universal adversarial perturbations aim to generate adversarial perturbations that are independent of specific images, and can mislead deep learning models with a high probability regardless of the original samples. According to the degree of dependence of the attack algorithm on different data distributions, universal adversarial perturbation (UAP) attacks can be divided into data-dependent, relaxed data-independent, and completely data-independent types. However, the prior art lacks a universal unified robustness evaluation method in a variety of data-dependent scenarios, and the evaluation algorithm capabilities are often unsatisfactory.

[0052] Based on the above problems, this embodiment provides an evaluation method for a deep learning model. Figure 2 is a flow chart of a method for evaluating a deep learning model according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0053] Step S210, obtaining an original sample, and obtaining an initial adversarial sample by adding a perturbation term to the original sample.

[0054] The above-mentioned original sample refers to the sample data set used to train the deep learning model to be evaluated; the sample can be an image, text, audio, etc., depending on the application field of the deep learning model. In order to detect the adversarial attack performance of the deep learning model, in this step, an initial perturbation term is added to the original sample to generate an initial adversarial sample. The initial perturbation term can be randomly generated based on the norm constraint.

[0055] Specifically, after adding some tiny disturbance noise that is difficult for the human eye to detect to the original image, the prediction result of the model will change. Such disturbance noise is called adversarial disturbance. In other words, adversarial disturbance is a small change added to the model input that can mislead the model output; the process of adding adversarial disturbance samples to mislead the target deep learning model to output incorrect prediction results is called adversarial attack. When improving the robustness of the target network model against adversarial attacks, adversarial training is a common method, that is, by simulating the adversarial attacks that the target network model may encounter in the real environment, generating adversarial sample images, and then attacking the target network model through the adversarial sample images, and evaluating the security of the model based on the success rate of the attack. Therefore, the authenticity of the adversarial disturbance is particularly important.

[0056] In order to better understand the present application, the following is a description of the attack scenario.

[0057] First, regarding the attacker’s goal definition: given a target deep learning model and a set of benign samples (i.e., the original samples mentioned above), the attacker attempts to find a universal adversarial perturbation δ so that most adversarial samples generated by the same perturbation δ can successfully mislead the deep learning model and produce incorrect predictions. The attacker uses L ∞ The perturbation amplitude is controlled by the norm constraint so that it is less than ∈, that is, ||δ||≤∈.

[0058] Secondly, regarding the attacker’s ability definition: the attacker cannot access the structure or weights of the target model, but only understands its classification task. In addition, the attacker can obtain a series of white-box alternative models that perform the same task. The attacker can obtain a certain data distribution P. X The attacker considers a variety of data dependency scenarios, such as strict data dependency, relaxed no data dependency, and completely no data dependency. Under these settings, the attacker can access different data distributions P X , such as the training data set Alternative datasets Or just access the human-designed Jigsaw dataset

[0059] Step S220, input the original sample into the integrated learning model, use each sub-model in the integrated learning model to process the original sample, obtain the first output value corresponding to each sub-model, and input the initial adversarial sample into the integrated learning model, use each sub-model to process the initial adversarial sample, and obtain the second output value corresponding to each sub-model.

[0060] The ensemble learning model includes multiple base learners (i.e., the above-mentioned sub-models), such as decision trees, neural network models, etc.; and the types and functions of the multiple sub-models in the ensemble learning model are different. Each original sample is input into each sub-model in the ensemble learning model, and for each sub-model, its output is calculated to obtain the first output value corresponding to each sub-model. Similarly, the initial Yang sample is input into each sub-model in the ensemble learning model, and the output of each sub-model is calculated to obtain the second output value.

[0061] In an optional embodiment, the first output value is the logical value output by each sub-model in the integrated learning model for the original sample; the second output value is the logical (Logits) value output by each sub-model in the integrated learning model for the initial adversarial sample. The output logical value (usually a probability value between 0 and 1) actually reflects the model's confidence that the input sample belongs to a certain category; this probabilistic interpretation enables the model to more intuitively understand the distribution characteristics of the input sample. Based on the two types of Logits as the optimization target in the subsequent steps, the model's perception of the input sample distribution can be maximized and the perturbation transferability can be improved.

[0062] It should also be noted that for the first output value and the second output value of each sub-model for the original sample and the initial adversarial sample, the integrated Logits value on the ensemble learning model can be calculated. The integration method can usually be an average or weighted summation method. For example, the original sample x in the ensemble learning model f i The Logits on Adversarial sample x+δ in ensemble learning model f i The Logits on

[0063]

[0064] Step S230, calculating the loss function result between the first output value and the second output value; based on the loss function result, optimizing the disturbance term and generating a target adversarial sample.

[0065] The above loss function is usually used to measure the difference between the first output value and the second output value. For example, cross entropy loss, mean square error, etc. can be used to calculate the above loss function result.

[0066] Specifically, based on the loss function result, an optimization algorithm (such as gradient descent, genetic algorithm, etc.) is used to adjust the perturbation term to generate a target adversarial sample that can maximize the loss function value. This process is usually an iterative process, and the iteration stops when a certain stopping condition is reached (such as the loss function value no longer changes significantly, the maximum number of iterations is reached, etc.).

[0067] Step S240: Use the target adversarial sample to attack the deep learning model to be evaluated, and generate an evaluation result for the deep learning model.

[0068] The generated target adversarial sample is input into the deep learning model to be evaluated to see whether the prediction result of the deep learning model is misleading. Finally, the evaluation result of the deep learning model is generated based on the prediction result of the model; the evaluation result may include indicators such as the accuracy, robustness, and success rate of adversarial attacks of the deep learning model.

[0069] The deep learning model evaluation method provided in the embodiment of the present application optimizes the disturbance term through the integrated learning model, and evaluates the deep learning model to be evaluated by the target adversarial sample that introduces the optimized disturbance term, so that the perturbation term of the adversarial sample can be applicable to a variety of deep learning models, thereby improving the universality of the adversarial sample; at the same time, the above-mentioned target adversarial sample is based on the output result of the integrated learning model, and the loss function result between the first output value of the original sample and the second output value of the adversarial sample is iteratively optimized and generated, without the need to use the true label of the original sample data, thereby removing the dependence on the true label, so that the target adversarial sample can be applicable to a variety of data-dependent scenarios, and there is no need to generate adversarial perturbations for specific images one by one based on the semantic content of the input sample, thereby effectively solving the problem of strong data dependence of the evaluation algorithm of the deep learning model and improving the evaluation efficiency of the deep learning model.

[0070] In some of these examples, see Figure 3 , the above calculation of the loss function result between the first output value and the second output value may further include the following steps:

[0071] Step S310, calculating the similarity result between the first output value and the second output value corresponding to each sub-model.

[0072] The similarity calculation can be based on a variety of methods, such as cosine similarity, Euclidean distance, Manhattan distance, Pearson correlation coefficient, etc. One of the calculation methods of the similarity result is shown in the formula: Cosine(f ei (x),f ei (x+δ)); in this formula, Cosine() is the cosine similarity measurement function, f ei(x) is the first output value of the i-th sub-model for the original sample x, f ei (x+δ) is the second output value of the i-th sub-model for the initial adversarial sample x+δ.

[0073] Step S320, obtain the preset loss weight parameters assigned to each sub-model; calculate the loss function result based on the loss weight parameters and the similarity result.

[0074] Before starting training, the loss weight parameters need to be initialized for each sub-model. These parameters can be determined by random initialization, manual setting, or based on some heuristic methods. For example, you can simply set all weights to the same value (such as 1 / N, where N is the number of sub-models), or assign different weights based on the importance of the task. During training, you can optimize the performance of the model by calculating the loss function and adjusting the loss weight parameters; this usually involves an iterative process. For example, the above loss function calculation process can be shown as follows:

[0075]

[0076] In the above formula, W i Used to represent the loss weight parameter corresponding to the i-th sub-model, where N is the number of sub-models; The constraint used to express this formula is that the sum of the N loss weight parameters is equal to 1.

[0077] Through the above steps S310 to S320, the similarity between the sub-model outputs can be effectively considered during the training process, and the optimization direction of the integrated learning model can be adjusted according to the preset loss weight parameters, thereby helping to improve the performance, stability and generalization ability of the integrated learning model and effectively optimize the adversarial disturbance term.

[0078] In some embodiments, the above-mentioned optimization of the perturbation term based on the loss function result and generation of the target adversarial sample may also include the following steps:

[0079] By changing the loss weight parameter to maximize the loss function result, the optimized loss weight parameter is obtained; the optimized loss function result is iteratively calculated according to the optimized loss weight parameter; by changing the perturbation term to minimize the optimized loss function result, the optimized perturbation term is iteratively calculated; the optimized perturbation term is added to the original sample to obtain the target adversarial sample.

[0080] In the process of generating target adversarial samples, optimizing the perturbation term based on the result of the loss function is a core step. Specifically, the loss function result is first maximized by changing the loss weight parameter. In this step, by adjusting these weight parameters, an attempt is made to find a configuration that maximizes the result of the loss function so that the performance of the model on a given sample (such as classification accuracy) is as poor as possible, thereby generating more adversarial samples. The weight configuration calculated above is recorded as the optimized loss weight parameter. The optimized loss weight parameter is then continued to calculate the loss function result again.

[0081] After that, the loss function result is minimized by changing the disturbance term. In this step, the goal is to find a disturbance term (i.e., a small modification to the original sample) that can minimize the optimized loss function result. In other words, in this embodiment, the optimization goal is to find a small modification so that when this modification is applied to the original sample, the performance of the model will drop significantly (for example, for classification tasks, it makes the model more likely to misclassify the modified sample). This step is also iterative because it is necessary to keep trying different disturbance terms and observe their impact on the loss function.

[0082] Finally, the found optimized perturbation term is added to the original sample, which usually involves a small adjustment to each pixel or feature value of the original sample. The sample after adding the perturbation term is the target adversarial sample. This sample may be very similar to the original sample visually, but it can significantly deceive the deep learning model and make it make wrong predictions.

[0083] In summary, the process of generating target adversarial samples is a complex iterative process that involves adjusting loss weight parameters, calculating loss function results, finding optimized perturbations, and adding perturbations to the original samples. The goal of this process is to find an adversarial sample that can effectively deceive the deep learning model, thereby improving the effectiveness of the deep learning model evaluation.

[0084] In some embodiments, the above-mentioned maximizing the loss function result by changing the loss weight parameter to obtain the optimized loss weight parameter may also include the following steps:

[0085] When the disturbance term is fixed, the first gradient of the loss function result with respect to the loss weight parameter is calculated, and the first gradient is transferred back to the integrated learning model. The maximization iterative optimization is performed based on the preset first learning rate to obtain the optimized loss weight parameter.

[0086] Specifically, in this step, the change of the disturbance term is temporarily ignored and regarded as a constant or fixed value. The gradient of the loss function result with respect to the loss weight parameter is calculated. This gradient is called the first gradient. The gradient represents the speed and direction of the change of the loss function result with the loss weight parameter. The calculated first gradient is passed back to the ensemble learning model to propagate the error from the output layer back to the input layer in order to adjust the weight. Based on the preset first learning rate (a hyperparameter that controls the update step size), iterative optimization is performed with the goal of maximizing the loss function result. In this process, the loss weight parameter is continuously adjusted until the loss function result reaches a local maximum or a certain stopping condition is met. The above process can be expressed by the following formula:

[0087]

[0088] Among them, t is used to represent the tth iteration, W t Used to represent the loss weight parameter of the tth iteration, W t-1 It is used to represent the loss weight parameter of the tth iteration, α is used to represent the first learning rate, It is used to represent the gradient calculation. It can be seen that through the above steps, the maximum optimization of the inner layer is achieved.

[0089] The above method of minimizing the optimized loss function result by changing the disturbance term and iteratively calculating the optimized disturbance term may also include the following steps:

[0090] When the optimization loss weight parameter is fixed, the second gradient of the optimization loss function result for the disturbance term is calculated, and the second gradient is transferred back to the ensemble learning model. Minimization iterative optimization is performed based on the preset second learning rate to obtain the optimized disturbance term.

[0091] In this step, the loss weight parameters are fixed and optimized, and the gradient of the loss function result for the disturbance term is calculated. This gradient is called the second gradient. Similarly, the gradient represents the speed and direction of the change of the loss function result with the disturbance term. The calculated second gradient is passed back to the ensemble learning model, and iterative optimization is performed based on the preset second learning rate (another hyperparameter that controls the update step size), with the goal of minimizing the optimized loss function result. In this process, the disturbance term is continuously adjusted until the loss function result reaches a local minimum or meets a certain stopping condition. The above process can be expressed by the following formula:

[0092]

[0093] In the above formula, CLIP is used to represent the clipping function, and β is used to represent the second learning rate. It can be seen that through the above steps, the outer minimum optimization is achieved.

[0094] Through the above embodiments, alternating optimization of the inner and outer layers is achieved, and the loss weight parameters and the disturbance terms are adjusted iteratively to maximize the loss function result and minimize the optimized loss function result, thereby obtaining the final optimization parameters, which helps to improve the performance of the integrated learning model, especially in scenarios such as adversarial training or robustness optimization.

[0095] In some embodiments, the above-mentioned use of the target adversarial sample to attack the deep learning model to be evaluated to generate an evaluation result for the deep learning model may also include the following steps:

[0096] The original sample is input into the deep learning model, the original label is output, and the target adversarial sample is input into the deep learning model, the predicted label is output; the original label and the predicted label are compared to generate an evaluation result.

[0097] Specifically, calculate the jth original sample x j In the deep learning model f to be evaluated v The original label of , the calculation process is shown in the following formula:

[0098] y=argmax(f v (x i ))

[0099] In the above formula, y represents the original label obtained by the above calculation, and argmax() represents a function that finds the parameter that maximizes the function value among the function parameters.

[0100] And, calculate the jth target adversarial sample x j +δ′ in the deep learning model f v The predicted label y ia , δ′ represents the optimized disturbance term; the calculation process is shown in the following formula:

[0101] y ia = argmax(f v (x i +δ))

[0102] Next, we count the number of mismatches between the predicted labels of the target adversarial samples with the optimized perturbation term and the original labels in each original sample, and the proportion of the total number of original samples to evaluate the robustness of the deep learning model. This process can be expressed as the following formula:

[0103]

[0104] In the above formula, ASR is used to represent the attack success rate, and M is used to represent the total number of original samples.

[0105] Through the above embodiments, attacks on deep learning models and model performance evaluation based on target adversarial samples are realized, effectively improving the accuracy of model performance evaluation.

[0106] In some embodiments, the above-mentioned comparison of the original label and the predicted label to generate the evaluation result may also include the following steps:

[0107] Obtain first sample identifiers corresponding to multiple original samples and second sample identifiers corresponding to multiple target adversarial samples; determine the correspondence between each original sample and each target adversarial sample based on the first sample identifier and the second sample identifier; based on the correspondence, compare the original label and the predicted label to obtain the attack result of each target adversarial sample, and summarize the attack results to generate an evaluation result.

[0108] The above-mentioned first sample identifier refers to the unique identifier of the original sample, which is used to distinguish different original data points. The second sample identifier is the unique identifier of the target adversarial sample, which is also used to distinguish different adversarial samples. Specifically, the two types of representations can be obtained by extracting or generating these identifiers from the data set. For the original sample, this can usually be obtained directly from the metadata of the data set. For adversarial samples, if they are generated by modifying the original sample, an identifier corresponding to the original sample can be retained or created.

[0109] Afterwards, based on the first sample identifier and the second sample identifier, the relationship between each original sample and its corresponding target adversarial sample is determined. Specifically, a dictionary or database can be used to store these correspondences, or the target adversarial sample can be determined to correspond to the original sample by comparing whether the identification numbers are consistent. For each target adversarial sample, a model is used for prediction to obtain a predicted label. According to the correspondence determined above, the predicted label is compared with the corresponding original label (that is, the true label of the original sample) to determine whether the attack is successful. Finally, based on the comparison results, the number and proportion of successful attacks are counted, and an evaluation report is generated. The evaluation report may include key indicators such as attack success rate, false alarm rate, and missed alarm rate.

[0110] Through the above embodiments, by adding corresponding sample identifiers to the original samples and the target adversarial samples respectively, when evaluating the deep learning model, the corresponding original samples and the target adversarial samples can be quickly and accurately matched, which is beneficial to improving the efficiency and accuracy of the deep learning model evaluation.

[0111] The present application is described and illustrated below in conjunction with specific embodiments. Figure 4 is a schematic diagram of a disturbance optimization method according to an embodiment of the present application, such as Figure 4 As shown, the disturbance optimization method includes the following steps:

[0112] Add the adversarial perturbation (i.e., the perturbation term) to the original sample to generate an initial adversarial sample. Input the original sample and the initial adversarial sample into the ensemble learning model respectively. The ensemble learning model is a model constructed based on the ensemble strategy for sub-models such as Model 1, Model 2, and Model 3. Calculate the cosine similarity of the adversarial Logits values ​​and the original Logits values ​​output by each sub-model in the ensemble learning model, and calculate the loss function result based on the cosine similarity and the preset loss weight parameter, and then perform alternating iterative optimization of the inner and outer layers based on the loss function result to update the adversarial perturbation and finally generate the target adversarial sample.

[0113] Afterwards, the original labels of the original samples in the deep learning model to be evaluated and the predicted labels of the target adversarial samples in the deep learning model are calculated. Based on the comparison results between the original labels and the predicted labels, the attack success rate for the deep learning model is calculated, thereby reflecting the robustness of the deep learning model.

[0114] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0115] This embodiment also provides an evaluation device for a deep learning model, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0116] Figure 5 is a structural block diagram of an evaluation device for a deep learning model according to an embodiment of the present application, such as Figure 5 As shown, the device includes: an acquisition module 51, an integrated learning module 52, an optimization module 53 and an evaluation module 54; wherein:

[0117] An acquisition module 51 is used to acquire an original sample and acquire an initial adversarial sample by adding a perturbation term to the original sample;

[0118] The ensemble learning model 52 is used to input the original sample into the ensemble learning model, process the original sample using each sub-model in the ensemble learning model, and obtain the first output value corresponding to each sub-model, and input the initial adversarial sample into the ensemble learning model, process the initial adversarial sample using each sub-model, and obtain the second output value corresponding to each sub-model;

[0119] An optimization module 53 is used to calculate a loss function result between the first output value and the second output value; based on the loss function result, optimize the disturbance term and generate a target adversarial sample;

[0120] The evaluation module 54 is used to use the target adversarial sample to attack the deep learning model to be evaluated, and generate an evaluation result for the deep learning model.

[0121] In some embodiments, the optimization module 53 is further used to calculate the similarity result between the first output value and the second output value corresponding to each sub-model;

[0122] The optimization module 53 is also used to obtain the preset loss weight parameters assigned to each sub-model; based on the loss weight parameters and the similarity results, the loss function results are calculated.

[0123] In some embodiments, the optimization module 53 is further used to maximize the loss function result by changing the loss weight parameter to obtain an optimized loss weight parameter;

[0124] The optimization module 53 is also used to iteratively calculate the optimization loss function result according to the optimization loss weight parameter;

[0125] The optimization module 53 is also used to minimize the optimization loss function result by changing the disturbance term, and iteratively calculate the optimized disturbance term;

[0126] The optimization module 53 is also used to add the optimized perturbation term to the original sample to obtain the target adversarial sample.

[0127] In some embodiments, the optimization module 53 is further used to calculate the first gradient of the loss function result with respect to the loss weight parameter under the condition of a fixed disturbance term, and to reversely transfer the first gradient to the ensemble learning model, and to perform maximization iterative optimization based on a preset first learning rate to obtain the optimized loss weight parameter;

[0128] The above-mentioned optimization module 53 is also used to calculate the second gradient of the optimization loss function result for the disturbance term under the condition of fixing the optimization loss weight parameter, and transfer the second gradient back to the integrated learning model, and perform minimization iterative optimization based on the preset second learning rate to obtain the optimized disturbance term.

[0129] In some embodiments, the evaluation module 54 is further configured to input the original sample into the deep learning model, output the original label, and input the target adversarial sample into the deep learning model, and output the predicted label;

[0130] The evaluation module 54 is also used to compare the original label and the predicted label to generate an evaluation result.

[0131] In some embodiments, the evaluation module 54 is further configured to obtain first sample identifiers corresponding to a plurality of original samples and second sample identifiers corresponding to a plurality of target adversarial samples;

[0132] The evaluation module 54 is further configured to determine a correspondence between each original sample and each target adversarial sample based on the first sample identifier and the second sample identifier;

[0133] The evaluation module 54 is also used to compare the original label and the predicted label based on the corresponding relationship, obtain the attack result of each target adversarial sample, and summarize the attack result to generate an evaluation result.

[0134] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0135] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0136] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0137] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0138] S1, obtain the original sample, and obtain the initial adversarial sample by adding a perturbation term to the original sample.

[0139] S2: Input the original sample into the integrated learning model, use each sub-model in the integrated learning model to process the original sample, obtain the first output value corresponding to each sub-model, and input the initial adversarial sample into the integrated learning model, use each sub-model to process the initial adversarial sample, and obtain the second output value corresponding to each sub-model.

[0140] S3, calculating the loss function result between the first output value and the second output value; based on the loss function result, optimizing the perturbation term and generating a target adversarial sample.

[0141] S4, uses the target adversarial sample to attack the deep learning model to be evaluated and generates evaluation results for the deep learning model.

[0142] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0143] In addition, in combination with the deep learning model evaluation method in the above embodiment, the present application embodiment can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the deep learning model evaluation methods in the above embodiment is implemented.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0146] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for evaluating a deep learning model, characterized in that: The method comprises: Obtain an original sample, and obtain an initial adversarial sample by adding a perturbation term to the original sample; Input the original sample into the ensemble learning model, use each sub-model in the ensemble learning model to process the original sample, obtain the first output value corresponding to each sub-model, and input the initial adversarial sample into the ensemble learning model, use each sub-model to process the initial adversarial sample, and obtain the second output value corresponding to each sub-model; Calculating a loss function result between the first output value and the second output value; optimizing the perturbation term based on the loss function result to generate a target adversarial sample; The target adversarial sample is used to attack the deep learning model to be evaluated, and an evaluation result for the deep learning model is generated.

2. The evaluation method according to claim 1, characterized in that: The calculating a loss function result between the first output value and the second output value includes: Calculate the similarity result between the first output value and the second output value corresponding to each sub-model; Obtaining a preset loss weight parameter assigned to each of the sub-models; and calculating the loss function result based on the loss weight parameter and the similarity result.

3. The evaluation method according to claim 2, characterized in that: The step of optimizing the disturbance term based on the loss function result and generating a target adversarial sample includes: By changing the loss weight parameter, the loss function result is maximized to obtain an optimized loss weight parameter; Iteratively calculate the optimized loss function result according to the optimized loss weight parameter; Minimize the optimization loss function result by changing the disturbance term, and iteratively calculate the optimized disturbance term; The optimized perturbation term is added to the original sample to obtain the target adversarial sample.

4. The evaluation method according to claim 3, characterized in that: The step of maximizing the loss function result by changing the loss weight parameter to obtain an optimized loss weight parameter includes: When the disturbance term is fixed, a first gradient of the loss function result with respect to the loss weight parameter is calculated, and the first gradient is reversely transmitted to the ensemble learning model, and a maximization iterative optimization is performed based on a preset first learning rate to obtain the optimized loss weight parameter; The step of minimizing the optimization loss function result by changing the disturbance term and iteratively calculating the optimized disturbance term comprises: When the optimization loss weight parameter is fixed, the second gradient of the optimization loss function result with respect to the disturbance term is calculated, and the second gradient is reversely transferred to the ensemble learning model, and minimization iterative optimization is performed based on a preset second learning rate to obtain the optimization disturbance term.

5. The evaluation method according to claim 1, characterized in that: The using the target adversarial sample to attack the deep learning model to be evaluated to generate an evaluation result for the deep learning model includes: Input the original sample into the deep learning model, output the original label, and input the target adversarial sample into the deep learning model, output the predicted label; The original label and the predicted label are compared to generate the evaluation result.

6. The evaluation method according to claim 5, characterized in that: The comparing the original label and the predicted label to generate the evaluation result includes: Obtaining a first sample identifier corresponding to a plurality of the original samples and a second sample identifier corresponding to a plurality of the target adversarial samples; Based on the first sample identifier and the second sample identifier, determining a correspondence between each of the original samples and each of the target adversarial samples; Based on the corresponding relationship, the original label and the predicted label are compared to obtain the attack result of each target adversarial sample, and the attack results are summarized to generate the evaluation result.

7. The evaluation method according to any one of claims 1 to 6, characterized in that: The first output value is a logical value output by each of the sub-models in the integrated learning model for the original sample; the second output value is a logical value output by each of the sub-models in the integrated learning model for the initial adversarial sample.

8. A deep learning model evaluation device, characterized in that: include: An acquisition module, used to acquire an original sample and acquire an initial adversarial sample by adding a perturbation term to the original sample; An integrated learning model, used to input the original sample into the integrated learning model, use each sub-model in the integrated learning model to process the original sample, and obtain a first output value corresponding to each sub-model, and input the initial adversarial sample into the integrated learning model, use each sub-model to process the initial adversarial sample, and obtain a second output value corresponding to each sub-model; An optimization module, configured to calculate a loss function result between the first output value and the second output value; based on the loss function result, optimize the disturbance term to generate a target adversarial sample; An evaluation module is used to use the target adversarial sample to attack the deep learning model to be evaluated, and generate an evaluation result for the deep learning model.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the deep learning model evaluation method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the deep learning model evaluation method according to any one of claims 1 to 7 when running.

Images