A robustness evaluation method for deep learning models with soft label output based on ORS

Abstract

The present invention proposes a method for evaluating the robustness of a deep learning model based on ORS for soft-label output, which belongs to the technical field of robustness evaluation. It includes generating adversarial text, using the adversarial text to attack a target model, calculating the robustness score of the target model based on ORS, and evaluating the robustness. The adversarial text is generated based on a word importance framework, and the word importance framework includes a sorting stage and a perturbation stage. The importance score is calculated using the confidence on the true value label corresponding to the text. If the importance score is greater than 0, it means that the words in the text have a positive impact on the true value label, that is, the words are important words in the text, otherwise the words are unimportant words in the text; it provides a feasible solution for the robustness evaluation of the target model with soft labels as output, solves the problem of incomplete evaluation indicators in the current robustness evaluation method, and quantifies the robustness evaluation criteria of the deep learning model with soft labels as output.

Description

Technical Field

[0001] The present invention relates to an ORS-based soft-label output deep learning model robustness evaluation method, belonging to the technical field of robustness evaluation. Background Art

[0002] Deep learning technology is increasingly used in production and life. However, various mainstream deep neural networks (DNN) will show varying degrees of vulnerability when subjected to adversarial attacks. The deep learning model under adversarial attack is called the target model. Escape attack is one of the most common adversarial attacks. It acts on the test phase after the target model is trained. The adversary often achieves escape attack by constructing adversarial examples (AE). AE is a malicious sample obtained by adding tiny perturbations that are difficult for humans to detect to the original benign sample, which can trigger the wrong prediction of the deep learning model.

[0003] Most of the current adversarial text generation methods only study the robustness of the target model under adversarial attacks by measuring the success rate of the attack on the target model, while ignoring the proportion of adversarial texts that misclassify the target model with high confidence when the input is a soft label. For a target model whose output is a soft label, its output not only includes the predicted label, but also the corresponding confidence. The present invention incorporates the indicator of the proportion of high-confidence adversarial text into the evaluation category of the robustness of the target model, and proposes a robustness evaluation method for deep learning models with soft label output based on ORS, which proposes a more comprehensive robustness evaluation scheme for target models that output soft labels. Summary of the invention

[0004] A brief overview of the present invention is provided below in order to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.

[0005] In view of this, in order to solve the technical problem that the robustness evaluation index of the target model whose output is a soft label in the prior art is not fully considered, the present invention provides a robustness evaluation method for a deep learning model with soft label output based on ORS. The present invention is oriented to the target model whose output is a soft label. Under the condition of adversarial attack, the two indicators of attack success rate and proportion of high-confidence adversarial text are comprehensively considered to evaluate the robustness of the target model, quantify its robustness, and make a more intuitive comparison of the robustness of different models. The present invention introduces a new mathematical concept, the offset average difference (OAD), and uses a variety of adversarial text generation methods for Chinese text classification tasks to evaluate the attack effectiveness of the target model and the proportion of high-confidence adversarial text, and designs a robustness scoring method OAD-basedRobustness Score (ORS) for the target model with soft label output based on OAD. This method comprehensively considers the attack success rate and the proportion of high-confidence adversarial texts of 9 Chinese adversarial text generation methods based on word importance on the target model in the black-box scenario; it includes not only all mainstream Chinese adversarial text generation methods, but also methods migrated from English adversarial attack methods; ORS uses the attack success rate and the proportion of high-confidence adversarial texts to calculate the robustness score of the target model under each method based on OAD, and finally obtains the average robustness score of the target model under adversarial text attacks.

[0006] Solution 1: Generate adversarial text based on the ORS soft-label output deep learning model robustness evaluation method, use the adversarial text to attack the target model, calculate the robustness score of the target model based on ORS, and evaluate the robustness.

[0007] Preferably, the adversarial text is generated based on a word importance framework, which includes a ranking stage and a perturbation stage.

[0008] Preferably, the ranking stage scores the importance of each word in the text and sorts the words from high to low according to the importance score, and the perturbation stage adds perturbations to the words in sequence.

[0009] Preferably, the method for scoring the importance of each word in the text is: calculate the importance score using the confidence on the true value label corresponding to the text; if the importance score is greater than 0, it means that the word in the text has a positive impact on the true value label, that is, the word is an important word in the text, otherwise the word is an unimportant word in the text; select all words with an importance score greater than 0, and sort the words from high to low according to the importance score.

[0010] Preferably, the perturbation method for sequentially adding perturbations to the words is one or any combination of the following perturbation methods:

[0011] Perturbation method 1: Convert simplified Chinese characters in the text into corresponding traditional Chinese characters;

[0012] Perturbation method 2: randomly select a Chinese character in the original text and directly replace it with the corresponding pinyin form;

[0013] Perturbation method 3: split a Chinese character into its radical components;

[0014] Perturbation method 4: Use a similar Chinese character to replace the original Chinese character;

[0015] Perturbation method 5: shuffle the order of Chinese characters contained in a Chinese word;

[0016] Perturbation method 6: perturb the original word into its homophonic word based on its Chinese pinyin;

[0017] Perturbation method seven: Find synonyms of the original word based on the synonym dictionary constructed by experts, and replace the original word with the selected synonym;

[0018] Perturbation method eight: Find the first k word vectors closest to the word vector of the original word in the word embedding space, and use the words corresponding to the word vectors as candidate replacement words for the original word;

[0019] Perturbation method nine: Use the MLM mechanism in the BERT model to generate candidate replacement words for the original words based on the text context.

[0020] Preferably, when perturbations are added to words in sequence, a plurality of candidate replacement words constitute a candidate word set, and when a word is selected from the candidate word set to replace the original word, the word with the largest change in target model confidence before and after the replacement is selected to replace the original word.

[0021] Preferably, the robustness score of the target model is calculated using ORS, and the robustness evaluation method is: quantifying the average deviation of a set of discrete real numbers from a given normal number, given a set of discrete values ​​containing n real numbers and a normal number, calculating the OAD value λ of the discrete data and the normal number c, where OAD is the average deviation of the deviation;

[0022] Calculate the i-th (i=1,...,n) target model facing the j-th (j=1,...,m) attack at any perturbation rate α k The proportion of high-confidence adversarial text under ij (α k );

[0023] When calculating the OAD value λ of discrete data and a positive constant c, an amplification factor β for the proportion of high-confidence adversarial text is introduced v And hyperparameter q; calculate the OAD value λ of the i-th target model under the j-th attack ij ;

[0024] Introduce a hyperparameter η to control the score system; calculate the robustness score rsv of the i-th target model to the proportion of high-confidence adversarial text under the j-th attack ij ;

[0025] Introduce the decision function H(z) and give the parameter β v Find a value where the independent variable z is β v , H(z) is the λ with the smallest OAD value among all attack methods for all models. ij (z), substitute z into H(z) starting from 1. If the calculated H(z) is greater than 0, add 1 to z and continue substituting it into H(z) until the value of H(z) is no greater than 0, that is, β v The value of is z minus 1. If the value of H(z) is still greater than 0 when z reaches 100, then the final β v The value of is 100;

[0026] Calculate any perturbation rate α k The attack success rate U ij (α k ), and set the arbitrary perturbation rate α k The proportion of high-confidence adversarial text under ij (α k ) is replaced by any perturbation rate α k The attack success rate U ij (α k ) to obtain the expansion coefficient β for the attack success rate u , then the overall expansion coefficient β is taken as the expansion coefficient β for the attack success rate u and the expansion factor β for the proportion of high-confidence adversarial text v The minimum value of

[0027] The model robustness score rsu based on the attack success rate is calculated using the overall expansion coefficient β value ij And the model robustness score rsv calculated based on the proportion of high confidence adversarial text ij , then the final robustness score rs of the i-th target model under the j-th attack ij is the model robustness score rsu calculated based on the attack success rate ij And the model robustness score rsv calculated based on the proportion of high confidence adversarial text ij The average value of the sum; if at a certain disturbance rate α k When the jth method is used to attack the i-th model, no adversarial text is successfully generated. At this time, the total number of adversarial texts successfully generated using the j-th attack method for the i-th target model is 0, then V ij (α k) cannot be calculated. In this case, the overall expansion coefficient β and the final robustness score rsv of the i-th target model under the j-th attack are calculated. ij When the total number of perturbation rate types p is replaced by V ij (α k ) calculates the number of perturbation rates corresponding to the actual value; if the jth method is used to attack the i-th target model, V under all perturbation rates ij (α k ) cannot be calculated, then when calculating β v When , the jth method for the i-th target model is skipped and rsv ij is set to the unrepresentable value NaN; in this case, the final robustness score rs ij By rsu ij Determine the final robustness score rs of the i-th target model under the j-th attack method ij is the model robustness score calculated based on the attack success rate.

[0028] Solution 2: An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for evaluating the robustness of a deep learning model based on ORS for soft label output as described in Solution 1 are implemented.

[0029] Solution 3: A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for robustness evaluation of deep learning models based on ORS for soft label output as described in Solution 1.

[0030] The beneficial effects of the present invention are as follows: the present invention proposes a robustness evaluation method for a deep learning model with soft label output based on ORS, which provides a feasible solution for the robustness evaluation of a target model with soft label output; the method can perform a quantifiable evaluation on the robustness of the target model so as to compare the robustness of different models under the same evaluation criteria; the method incorporates the attack success rate and the proportion of high-confidence adversarial texts into the evaluation index of the robustness of the target model in view of the characteristics of soft label output, so as to more comprehensively evaluate its robustness; the method introduces an expansion coefficient β, so as to expand the gap in the robustness scores between different models as much as possible within the allowable range, so as to more intuitively evaluate the robustness of the target model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0032] Figure 1 Flowchart of the robustness evaluation method for deep learning models based on ORS for soft label output. DETAILED DESCRIPTION

[0033] In order to make the technical solutions and advantages of the embodiments of the present invention more clearly understood, the exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0034] Example 1: Reference Figure 1 This embodiment describes a method for evaluating the robustness of deep learning models based on ORS soft-label output, generates adversarial text, uses the adversarial text to attack a target model, and calculates the robustness score of the target model based on ORS to evaluate the robustness.

[0035] The present invention proposes a new concept of Offset Average Difference (OAD), which is the average value of the sum of the differences between a set of discrete real numbers and a normal number, and is used to measure the average deviation between the set of real numbers and the normal number.

[0036] The present invention proposes an OAD-Based Robustness Score (ORS) to evaluate the robustness of the target model.

[0037] Adversarial text is generated based on a word importance framework, which includes a ranking stage and a perturbation stage.

[0038] In the sorting phase, the importance of each word in the text is scored and the words are sorted from high to low according to their importance scores. In the perturbation phase, perturbations are added to the words one by one.

[0039] The method for scoring the importance of each word in the text is: use the confidence on the true value label corresponding to the text to calculate the importance score. If the importance score is greater than 0, it means that the word in the text has a positive impact on the true value label, that is, the word is an important word in the text, otherwise the word is an unimportant word in the text; select all words with an importance score greater than 0 and sort the words from high to low according to the importance score.

[0040] Given a text x, which contains n words, that is, x = {w1, w2, ..., w n}, assuming that the true value label corresponding to the text x is y, that is, f(x) = y, and use f y (x) represents the confidence of text x on label y, then any word w in x i(i=1,...,n), its importance score I(w i ) is expressed as:

[0041]

[0042] Among them, x\w i Indicates that the word w i Delete from the text x and calculate the importance score of each word in the text. If I(w i )>0, then the word w i It has a positive impact on the true value label y, that is, w i is an important word in x, otherwise it is a non-important word in x; select all words with importance scores greater than 0, and sort the words from high to low according to the importance scores, so as to facilitate the subsequent addition of disturbances to these words in sequence.

[0043] Adding perturbation methods to the words in sequence is one or any combination of the following perturbation methods;

[0044] Perturbation method 1: Convert simplified characters in the text into corresponding traditional characters; the writing form of Chinese characters has been changing with the evolution of the times, and traditional characters are one of the most widely known forms of Chinese characters. Currently, most documents are written in simplified characters. After using perturbation method 1, the purpose of adding perturbations can be achieved to a certain extent; this perturbation method is suitable for Chinese characters with different simplified and traditional forms.

[0045] Perturbation method 2: randomly select a Chinese character in the original text and directly replace it with the corresponding pinyin form; pinyin is a Chinese-specific spelling rule that helps humans understand the meaning of the original text, but it is easy to automatically correct the adversarial text by restoring pinyin to text.

[0046] Perturbation method three: split a Chinese character into its components; Chinese characters are a language that can express both sound and meaning, and its meaning is often expressed through its components. After splitting a Chinese character by components, its meaning can still be understood. However, in order to ensure the fluency of the adversarial text, this embodiment only splits the Chinese characters with left and right structures.

[0047] Perturbation method 4: Use a similar Chinese character to replace the original Chinese character; visualize the text and find a similar Chinese character through CNN.

[0048] Perturbation method five: shuffle the order of Chinese characters contained in a Chinese word; a method of replacing adjacent letters in English and migrating them to Chinese.

[0049] Perturbation method six: perturb the original word into its homophonic word based on its Chinese pinyin; homophonic words mainly include homophones, homophonic words with different front and back nasal sounds, and homophonic words with different flat and retroflex tongue sounds.

[0050] Perturbation method seven: Find synonyms of the original word based on the synonym dictionary constructed by experts, and replace the original word with the selected synonym;

[0051] Perturbation method eight: Find the first k word vectors closest to the word vector of the original word in the word embedding space, and use the words corresponding to the word vectors as candidate replacement words for the original word; the words found in the word embedding space are likely to be the antonyms of the original words. For tasks such as sentiment classification, antonyms may completely change the semantics of the original sentence. Therefore, when generating adversarial text for such tasks, it is necessary to filter the antonyms of the original words first. Use the sentiment vocabulary ontology dictionary to filter antonyms. Filter candidate words that are inconsistent with the original word's part of speech.

[0052] Perturbation method nine: Use the MLM mechanism in the BERT model to generate candidate replacement words for the original word based on the text context; improve the fluency of the adversarial text, limit the length of the original word to 1, and the length of its replacement word to 1; and when the length of the original word is not less than 2, the length of its replacement word is 2; antonyms also need to be filtered when facing the sentiment tendency classification task.

[0053] When perturbations are added to words in sequence, there are multiple candidate replacement words constituting a candidate word set. When a word is selected from the candidate word set to replace the original word, the word with the largest change in the confidence of the target model before and after the replacement is selected to replace the original word.

[0054] Given a text x containing n words, n} and a word w in x i (i=1,...,n), assuming w i Candidate word set For the candidate word set Any candidate word c in j (j=1,...,m), confidence change C(c j ) is calculated using the formula:

[0055]

[0056] in, Indicates that the word w i Replace with candidate word c j The final replacement word w is i 'express:

[0057]

[0058] This embodiment uses adversarial text to attack the target model based on transferability to attack ChatGPT. ChatGPT is not a professional classification model and can only output hard labels without confidence information. Therefore, this embodiment indirectly attacks it based on the transferability of adversarial text, and attacks the Chinese BERT output as a soft label as an intermediate target model. The confidence information output by it is used to construct the adversarial text using the adversarial text generation method based on word importance, and then the adversarial text generated for Chinese BERT is used to attack the final target model ChatGPT. For text classification tasks, the ChatGPT model of gpt-3.5-turbo-0613 version is used as the target model, and its API is called to attack it; for different classification tasks, the prompts used are different, specifically:

[0059] When faced with the news classification task, the prompt used is as follows: "Please tell me which category the following news belongs to: science and technology news, education news, financial news, social news, or sports news. When answering, please use 0 for science and technology news, 1 for education news, 2 for financial news, 3 for social news, 4 for sports news, and 5 for unclassifiable news. When answering, please directly give the number corresponding to the category without explaining the reason."

[0060] For the sentiment classification task, the prompt used in this paper is as follows: "Please help me determine the sentiment polarity of the following text. Use 0 for negative sentiment, 1 for positive sentiment, and 2 for undetermined sentiment (only give the result without explanation, and only give one label for the entire text, no need to give multiple labels)."

[0061] Under the condition that ChatGPT does not classify the adversarial text as unclassifiable, if ChatGPT classifies the adversarial text as a label inconsistent with its true value label, it means that ChatGPT is deceived by the adversarial text and the attack is considered successful.

[0062] The robustness score of the target model is calculated based on ORS. The robustness evaluation is based on the OAD target model robustness scoring method ORS. The robustness of the target model is quantifiable. This method can not only evaluate the robustness of a target model in the face of different adversarial attacks, but also compare the robustness of multiple target models under multiple adversarial attack conditions on the same benchmark, and maximize the gap in the robustness scores of the target model in the face of different adversarial attacks, which is convenient for more intuitive comparison. The specific method is: OAD quantifies the average deviation of a set of discrete real numbers from a given normal number. Formal expression: Given a set of discrete values ​​x1, x2, ... x containing n real numbers n and a positive constant c (c>0), calculate the OAD value λ of the discrete data and the positive constant c, the calculation method is:

[0063]

[0064] OAD is used to calculate the robustness score of the target model whose output is a soft label. The proportion of high-confidence adversarial texts under each perturbation ratio is used as a set of discrete real numbers in formula (4). There are n target models, and each target model has m attack methods. For each attack method, there are p perturbation rates. For any perturbation rate, α is used k (k=1,...,p); for the i-th (i=1,...,n) target model, when facing the j-th (j=1,...,m) attack, the proportion of high-confidence adversarial texts at different perturbation rates can be expressed as V ij (α1),V ij (α2),...,V ij (α p ) means that at any disturbance rate α k The proportion of high-confidence adversarial text under V ij (α k ) indicates that V ij (α k ) can be calculated using the following formula:

[0065]

[0066] where v ij (α k ) means that when the disturbance rate is α k The number of adversarial texts misclassified with high confidence when . And T′ represents the total number of adversarial texts successfully generated using the jth attack method for the i-th target model.

[0067] In order to make the difference in robustness scores between different attack methods more obvious, an amplification factor β for the proportion of high-confidence adversarial text is introduced into formula (4): v ; In order to facilitate the calculation of the final robustness score, β v Set to a positive integer, at the same time, in order to set the parameter β v The value range is controlled in (0,100], a hyperparameter q is introduced, and the q value is set to 100;

[0068] Calculate the OAD value λ of the i-th target model under the j-th attack ij , the calculation method is:

[0069]

[0070] In order to ensure that the final robustness score is in the range of (0,10], a hyperparameter η is introduced to control the score system; the value of constant c and hyperparameter q is set to 10;

[0071] Calculate the robustness score rsv of the i-th target model under the j-th attack with respect to the proportion of high-confidence adversarial text ij , the calculation method is:

[0072]

[0073] Introduce the decision function H(z) and give the parameter β v Find a value where the independent variable z is β v , H(z) is the λ with the smallest OAD value among all attack methods for all models. ij (z), substitute z into H(z) starting from 1. If the calculated H(z) is greater than 0, add 1 to z and continue substituting it into H(z) until the value of H(z) is no greater than 0, that is, β v The value of is z minus 1. If the value of H(z) is still greater than 0 when z reaches 100, then the final β v The value of is 100.

[0074] The specific method is as follows: when the robustness score range is (0,10], the robustness score of each target model facing each attack method is widened to make the score comparison more intuitive; in order to find this value, a decision function H(z) is introduced, whose independent variable z is the value of βv; with λ ij Let H(z) be represented as:

[0075]

[0076] Parameter β v The value of is expressed using the decision function H(z):

[0077]

[0078] Combining formula (8) and formula (9), we get the expansion coefficient β for the proportion of high-confidence adversarial text v ;

[0079] Calculate the attack success rate U under each disturbance ratio ij (α k ), replace V in formulas (6)-(8) ij (α k ) is replaced by U ij (α k ), and calculate the expansion coefficient β for the attack success rate u , then the overall expansion coefficient β takes the value:

[0080] β=min{β u ,β v} (10)

[0081] Substitute the final β value into formula (7) and replace β v , we get the robustness score rsv based on the proportion of high confidence adversarial text ij ; Substitute the final β value into formula (7) and replace β v , and replace the parameter V in the formula ij (α k ) is replaced by U ij (α k ), and obtain the model robustness score rsu calculated based on the attack success rate ij , then the final robustness score rs of the i-th target model under the j-th attack ij Use the following formula to calculate:

[0082] rs ij =(rsu ij +rsv ij ) / 2 (11)

[0083] If at some disturbance rate α k When the jth method is used to attack the i-th model, no adversarial text is successfully generated. If T' in the formula is 0, then V ij (α k ) cannot be calculated, in this case, calculate the parameter β v and rsv ij When , the total number of perturbation rate types p is replaced by the number of perturbation rates corresponding to the actual value calculated by Vij(αk); if the jth method is used to attack the i-th target model, V under all perturbation rates ij (α k ) cannot be calculated, then when calculating β v When , the jth method for the i-th target model is skipped and rsv ij is set to the unrepresentable value NaN; in this case, the final robustness score rs ij By rsu ij The final robustness score rs of the i-th target model under the j-th attack method is ij It is expressed as:

[0084] rs ij =rsu ij (12)

[0085] Embodiment 2: The computer device of the present invention may be a device including a processor and a memory, such as a single-chip microcomputer including a central processing unit. Furthermore, the processor is used to implement the steps of the above-mentioned ORS-based soft label output deep learning model robustness evaluation method when executing the computer program stored in the memory.

[0086] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0087] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0088] Embodiment 3: Computer readable storage medium embodiment.

[0089] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by a processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. A computer program is stored on the computer-readable storage medium. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned ORS-based soft label output deep learning model robustness evaluation method can be implemented.

[0090] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0091] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative, not restrictive, with respect to the scope of the present invention, which is defined by the appended claims.

Claims

1. A robustness evaluation method for deep learning models with soft label output based on ORS, characterized by: Generate adversarial text and use it to attack the target model. Use ORS to calculate the robustness score of the target model and evaluate the robustness. The robustness score of the target model is calculated using ORS. The robustness evaluation method is as follows: quantify the average deviation of a set of discrete real numbers from a given normal number. Given a set of discrete values ​​containing n real numbers and a normal number, calculate the OAD value λ between the discrete data and the normal number c, where OAD is the average deviation of the deviation; Calculate the i-th (i=1,...,n) target model facing the j-th (j=1,...,m) attack at any perturbation rate α k The proportion of high-confidence adversarial text under ij (α k ); When calculating the OAD value λ of discrete data and a positive constant c, an amplification factor β for the proportion of high-confidence adversarial text is introduced v And hyperparameter q; calculate the OAD value λ of the i-th target model under the j-th attack ij ; Introduce a hyperparameter η to control the score system; calculate the robustness score rsv of the i-th target model to the proportion of high-confidence adversarial text under the j-th attack ij ; Introduce the decision function H(z) and give the parameter β v Find a value where the independent variable z is β v , H(z) is the λ with the smallest OAD value among all attack methods for all models. ij (z), substitute z into H(z) starting from 1. If the calculated H(z) is greater than 0, add 1 to z and continue substituting it into H(z) until the value of H(z) is no greater than 0, that is, β v The value of is z minus 1. If the value of H(z) is still greater than 0 when z reaches 100, then the final β v The value of is 100; Calculate any perturbation rate α k The attack success rate U ij (α k ), and set the arbitrary perturbation rate α k The proportion of high-confidence adversarial text under ij (α k ) is replaced by any perturbation rate α k The attack success rate U ij (α k ) to obtain the expansion coefficient β for the attack success rate u , then the overall expansion coefficient β is taken as the expansion coefficient β for the attack success rate u and the expansion factor β for the proportion of high-confidence adversarial text v The minimum value of The model robustness score rsu based on the attack success rate is calculated using the overall expansion coefficient β value ij And the model robustness score rsv calculated based on the proportion of high confidence adversarial text ij , then the final robustness score rs of the i-th target model under the j-th attack ij is the model robustness score rsu calculated based on the attack success rate ij And the model robustness score rsv calculated based on the proportion of high confidence adversarial text ij The average of the sum; If at some disturbance rate α k When the jth method is used to attack the i-th model, no adversarial text is successfully generated. At this time, the total number of adversarial texts successfully generated using the j-th attack method for the i-th target model is 0, then V ij (α k ) cannot be calculated. In this case, the overall expansion coefficient β and the final robustness score rsv of the i-th target model under the j-th attack are calculated. ij When the total number of perturbation rate types p is replaced by V ij (α k ) The number of disturbance rates corresponding to the actual value is calculated; If the jth method is used to attack the i-th target model, V ij (α k ) cannot be calculated, then when calculating β v When , the jth method for the i-th target model is skipped and rsv ij is set to the unrepresentable value NaN; in this case, the final robustness score rs ij By rsu ij Determine the final robustness score rs of the i-th target model under the j-th attack method ij is the model robustness score calculated based on the attack success rate.

2. The method for evaluating the robustness of deep learning models based on ORS and soft label output according to claim 1 is characterized in that: Adversarial text is generated based on a word importance framework, which includes a ranking stage and a perturbation stage.

3. The method for evaluating the robustness of deep learning models based on ORS and soft label output according to claim 2 is characterized in that: In the sorting phase, the importance of each word in the text is scored and the words are sorted from high to low according to their importance scores. In the perturbation phase, perturbations are added to the words one by one.

4. The method for evaluating the robustness of a deep learning model based on ORS and soft label output according to claim 3 is characterized in that: The method for scoring the importance of each word in the text is: use the confidence on the true value label corresponding to the text to calculate the importance score. If the importance score is greater than 0, it means that the word in the text has a positive impact on the true value label, that is, the word is an important word in the text, otherwise the word is an unimportant word in the text; select all words with an importance score greater than 0 and sort the words from high to low according to the importance score.

5. The method for evaluating the robustness of deep learning models based on ORS and soft label output according to claim 4 is characterized in that: Adding perturbation methods to the words in sequence is one or any combination of the following perturbation methods; Perturbation method 1: Convert simplified Chinese characters in the text into corresponding traditional Chinese characters; Perturbation method 2: randomly select a Chinese character in the original text and directly replace it with the corresponding pinyin form; Perturbation method 3: split a Chinese character into its radical components; Perturbation method 4: Use a similar Chinese character to replace the original Chinese character; Perturbation method 5: shuffle the order of Chinese characters contained in a Chinese word; Perturbation method 6: perturb the original word into its homophonic word based on its Chinese pinyin; Perturbation method seven: Find synonyms of the original word based on the synonym dictionary constructed by experts, and replace the original word with the selected synonym; Perturbation method eight: Find the first k word vectors closest to the word vector of the original word in the word embedding space, and use the words corresponding to the word vectors as candidate replacement words for the original word; Perturbation method nine: Use the MLM mechanism in the BERT model to generate candidate replacement words for the original words based on the text context.

6. The method for robustness evaluation of deep learning models based on ORS and soft label output according to claim 5 is characterized in that: When perturbations are added to words in sequence, there are multiple candidate replacement words constituting a candidate word set. When a word is selected from the candidate word set to replace the original word, the word with the largest change in the confidence of the target model before and after the replacement is selected to replace the original word.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for robustness evaluation of a deep learning model based on ORS for soft label output according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for evaluating the robustness of a deep learning model based on ORS and oriented to soft label output as described in any one of claims 1 to 6 is implemented.

Images