An attack defense method for large character set verification code based on an adversarial sample

Abstract

The application discloses a large character set verification code attack defense method based on an adversarial sample, first, existing large character set verification code data on a Chinese verification code service website is collected and manually labeled, and extremely similar new verification codes are generated by imitating the crawled verification codes, so that an expanded verification code data set is constructed; then, a character detection and recognition model is constructed, and noise is generated on the characters and the background of the verification code based on the two types of models; finally, according to an adversarial sample verification code generation method combining input transformation, gradient attack and an attention mechanism, adversarial sample characters aiming at the verification code character recognition model are generated; for the threat to the verification code character detection model, an integrated adversarial sample generation method is adopted, loss functions of multiple different architecture character detection models are integrated, a black box setting of the different architecture character detection models is converted into a white box setting, and adversarial noise is generated on the background of the verification code to defend the character detection model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to a large character set verification code attack defense method based on an adversarial sample. BACKGROUND

[0002] A verification code is a public automatic program used to distinguish whether a user is a computer or a person. A well-designed verification code mechanism should enable a human to pass the verification code test very easily, while excluding an automated cracking program written by an attacker. At present, verification codes have been widely applied to various online websites and application programs, and to some extent, prevent various malicious behaviors on the Internet, including malicious registration, spam, social robots, etc.

[0003] Although there are many types of verification codes designed at present, due to the low generation cost and high user acceptance, character type verification codes are still the most widely used and the most convenient type of verification code for users and websites. The character type verification code is generally designed as a plurality of difficult-to-identify characters plus a certain interference background, and requires the user to input the characters in the verification code.

[0004] Unfortunately, the character type verification code has been cracked by many research works. One of the reasons is that the traditional character type verification code only contains English letters and Roman numerals. In the case of a small character type verification code classification space, using the deep learning model popular in recent years will easily crack this type of verification code.

[0005] In order to solve this problem, a verification code with a larger character set is designed. The large character set verification code includes a verification code containing Chinese, Japanese and Korean characters. Taking the Chinese large character set verification code as an example, the Chinese large character set verification code usually contains 3755 commonly used Chinese characters. A larger character set increases the target category of the verification code attack model, thereby improving the security of the verification code. This type of verification code increases the difficulty of constructing and training the attacker model, and also reduces the success rate of cracking to some extent.

[0006] However, with the development of research work, the large character set verification code also faces serious security threats. In recent years, many works have analyzed the security of Chinese large character set verification codes and successfully cracked the Chinese large character set verification codes of many large websites with high accuracy. In order to defend against attacks on large character set verification codes, many methods proposed at present mostly focus on increasing the complexity of the verification code, but reduce the usability of the verification code. And with the continuous development of deep learning technology, they can still be cracked by more efficient methods. SUMMARY

[0007] In view of the above problems, the purpose of the present application is to provide an attack defense method for large character set verification code based on adversarial samples, which is different from the existing method of improving the complexity of the verification code, and based on the adversarial sample technology, the security of the existing large character set verification code on the real website is improved on the basis of protecting the availability of the verification code. The technical scheme is as follows:

[0008] An attack defense method for large character set verification code based on adversarial samples, comprising the following steps:

[0009] Step 1: Collecting existing large character set verification code data on Chinese verification code service website and manually labeling, and generating new verification code by imitation, so as to build an expanded verification code data set;

[0010] Step 2: Building character detection model and character recognition model, the character detection model includes a white box model for generating noise for the character detection model, and a black box model for testing the migration effect of the adversarial sample; the character recognition model includes a white box model for generating adversarial sample characters to defend the character recognition model, and a black box model for testing the migration effect of the adversarial sample; noise is generated on the characters and background of the verification code based on the character detection model and the character recognition model;

[0011] Step 3: Generation of adversarial sample verification code

[0012] Step 3.1: In the process of generating adversarial sample verification code, an adversarial sample generation method combining gradient attack, input transformation and attention mechanism is adopted to generate adversarial sample characters for the character recognition model of the verification code;

[0013] Step 3.2: In the face of the threat of the character detection model of the verification code, an integrated adversarial sample generation method is adopted, which integrates the loss functions of multiple different architecture character detection models, converts the black box settings of different architecture character detection models into white box settings, and generates adversarial noise on the background of the verification code to defend the character detection model.

[0014] Further, in step 1, the labeling information of the verification code data and the manual labeling includes: the position of the character in the verification code and the character category; first, count the size of the character set appearing in the collected verification code picture; then, divide each kind of labeled verification code into two parts, one part is used to train the verification code cracking model, and the other part is used to test the attack model and produce adversarial sample verification code.

[0015] Further, in step 2, the character detection model and the character recognition model further comprise:

[0016] 1) Character detection model selection Faster-RCNN, YOLO-v5 and SSD as the target detection architecture; for Faster-RCNN, train ResNet-50, ResNet-101 and VGG three kinds of model of main network; for SSD, train MobileNetV2 and VGG two kinds of model of main network; for YOLO-v5, use CSPDarknet-53 as the main network for training;

[0017] The label of the character detection model is set to whether there is a character, that is, the character detection model uniformly takes the character as the foreground during training and testing, without specific character category identification;

[0018] Faster-RCNN with ResNet-101 as the main network, SSD with MobileNetV2 as the main network, and YOLO-v5 as the white box model; Faster-RCNN with ResNet-50 and VGG as the main network, and SSD with VGG as the main network as the black box model;

[0019] 2) Character recognition model selection Network model Inception-Resnet-v2, Inception-v3, ResNet-50 and VGG-16 for single character multi-classification recognition; use Inception-Resnet-v2 as the white box model, and use Inception-v3, ResNet-50 and VGG-16 as the black box model.

[0020] Further, the step 3 generates the adversarial sample characters for the Captcha character recognition model, which specifically includes:

[0021] Step 3.1.1: Determine the loss function of the adversarial sample generation model f

[0022] Find the adversarial sample picture x adv Satisfy the following conditions

[0023] f(x adv ; θ)≠y (1)

[0024] ||x-x adv ||∞<ε (2)

[0025] Where, θ is the parameter of the adversarial sample generation model f, x is an arbitrary input character picture, y is the category true label of the picture x, for example, the label of the character picture "person" is "person" this word under the encoding y of the white box model, ε is the maximum noise amplitude at each pixel point;

[0026] The purpose of finding the adversarial sample is achieved by maximizing the loss value of the input character picture x to the real class under the adversarial sample generation model f, and the category cross entropy J is used as the loss function of the adversarial sample generation model f:

[0027]

[0028] Step 3.1.2: Gradient-based attack

[0029] The calculation is The derivative of the loss function In the gradient iteration step, the gradient variance reduction and Nesterov iteration are combined; before each iteration, the adversarial sample character x in the current epoch is updated once to obtain

[0030]

[0031] Wherein, is the adversarial sample character in the current epoch, u is the momentum attenuation factor, a is the noise amplitude added in each iteration step, is the updated adversarial sample character; g t is the gradient of the tth iteration;

[0032] In each epoch, after updating the gradient, the gradient variance v is updated according to the following formula t :

[0033]

[0034] Wherein, v t+1 is the updated gradient variance; and

[0035]

[0036] In the formula, N represents that the gradient of N adjacent points around the current x is sampled when calculating the gradient variance; the adjacent points of x are defined as x i =x+r i , r U [-(β·ε) d , (β·ε) d ], wherein U [-(β·ε) d , (β·ε) d ] represents a random vector, each point obeys uniform distribution in the range, β is a hyperparameter for gradient variance calculation, r i represents the ith random vector; r represents a random vector sampled from U [-(β·ε) d , (β·ε) d ]; ​denotes the derivative of the loss function with respect to x before random sampling, denotes the derivative of the loss function with respect to the neighboring point of x;

[0037] At the gradient update, the gradient variance v t and the derivative of the loss function is added, and the gradient g t of the t-th iteration is updated based on the momentum:

[0038]

[0039] where g t+1 is the gradient after momentum update, and 1 represents the L1 norm;

[0040] Step 3.1.3: input transformation attack

[0041] Multiple input methods are adopted: before the character picture is input into the adversarial sample generation model, the shaping and padding are applied to it with a probability p:

[0042] D(x; p) (8)

[0043] Translation invariance method is adopted: the picture is translated to achieve data augmentation, and the kernel matrix is used for convolution operation with the original picture:

[0044] T(g) = W * g (9)

[0045] where W is a predefined convolution kernel matrix, T(g) is the gradient after convolution operation, and g is the original gradient;

[0046] Scaling invariance method is adopted: a single input picture is expanded into a series of new pictures, and the weighted average of the gradients of these new pictures is taken when calculating the gradient:

[0047]

[0048] where m0 represents the number of expanded pictures;

[0049] Step 3.1.4: adjusting the noise amplitude based on attention mechanism on the verification code character

[0050] To obtain the contribution of each position in the white box model feature layer to the result, the attention of the adversarial sample generation model is approximated as a feature map the spatial domain pooling gradient of the true label y of the input picture x:

[0051]

[0052] where, Z is a normalization factor, such that The attention weight of the cth feature map of the kth convolution layer relative to the true label y is obtained f(x) represents a white box model; m and n represent the horizontal and vertical coordinates of a pixel point, respectively;

[0053] The different feature maps are scaled by the attention weight to obtain the features that the model pays more attention to, and then the features of each channel are summed and processed using a ReLu activation function to remove negative values to obtain

[0054]

[0055] wherein, represents the attention activation map after removing negative values;

[0056] The absolute value of the result of weighted summing the feature maps is taken, and the result is normalized to [0, 1] again, and then the obtained is reshaped to the same size as the input picture; due to the translation invariance of the convolutional neural network, the heat map mapped back represents the attention weight of the model at different positions in the original picture:

[0057]

[0058] wherein, represents the attention activation map after taking the absolute value;

[0059] According to an epsilon-based mask is constructed, which determines the noise amplitude that can be updated at each iteration during the iterative adversarial sample generation process when updating the noise of the adversarial sample:

[0060]

[0061] wherein, α is the noise update amplitude of each epoch, α = ε / T, ε is the maximum noise amplitude allowed at each point, T is the number of iterations, γ is a hyperparameter for amplifying the step size, and it is used to crop the picture at the end of the iteration to ensure that ||x-x adv || ∞ < ε.

[0062] Further, step 3.2 specifically includes:

[0063] The loss function used when training the three models Faster-RCNN, YOLO-v5, and SSD participating in the ensemble attack is used as the loss function for generating adversarial samples for the detection model;

[0064] The loss function L used when training Faster-RCNNfrcnn is:

[0065] L frcnn is: rpnCls +L rpnLoc +L roiCls +L roiLoc (15)

[0066] wherein, L rpnCls represents the classification loss of the RPN module of Faster-RCNN, L rpnLoc represents the Bounding Box regression loss of the RPN module of Faster-RCNN, used for training the positioning ability of the model; L roiCls represents the classification loss of the ROI module, L roiLoc represents the Bounding Box regression loss of the ROI module;

[0067] The loss function L yolov5 of YOLO-v5 during training is:

[0068] L yolov5 is: loc +L cls +L conf (16)

[0069] wherein, L loc is the position loss of YOLO-v5, L cls is the classification loss of whether to contain an object, L conf is the multi-class confidence loss;

[0070] The loss function L ssd of SSD during training is:

[0071] L ssd is: loc ′+L cls ′ (17)

[0072] wherein, L loc ′ is the position loss of SSD, L cls ′ is the classification loss;

[0073] In the process of integrated attack, the three losses are weighted and averaged based on different weights to obtain an integrated loss:

[0074] J ens (x,y; θ) = w1L frcnn (x,y; θ1) + w2L yolov5 (x,y; θ2) + w3L ssd (x,y; θ3) (18)

[0075] Wherein, θ1, θ2, θ3 respectively represent model parameters of Faster-RCNN, YOLO-v5, SSD; w1, w2, w3 respectively represent weights of three kinds of loss functions.

[0076] The beneficial effects of the present application are:

[0077] 1) The present application aims at the current situation that large character set verification codes are cracked based on deep learning models, and generates adversarial sample verification codes for defense by constructing character detection and recognition models and generating noise on the characters and background of the verification code based on the two types of models.

[0078] 2) The present application proposes an adversarial sample verification code generation method combining input transformation, gradient attack and attention mechanism, namely M-VNI-CT-FGSM, which can improve the generalization of adversarial samples and be used to generate adversarial sample characters against character recognition models of verification codes.

[0079] 3) The present application is based on an integrated adversarial sample generation method, namely SVRE-MI-FGSM, which integrates the loss functions of multiple different architecture character detection models, converts the black box settings of different architecture character detection models into white box settings, and generates adversarial noise on the background of the verification code to defend against the threat of character detection models.

[0080] 4) The present application has achieved remarkable results in generating adversarial sample verification codes to defend against verification code cracking attacks, and has improved the security of large character set verification codes on existing real websites on the basis of protecting the usability of verification codes, providing methods and ideas for future large character set verification code defense. BRIEF DESCRIPTION OF DRAWINGS

[0081] Figure 1 It is the basic framework diagram of the large character set verification code attack defense method based on adversarial samples of the present application.

[0082] Figure 2 It is the flowchart of M-VNI-CT-FGSM algorithm.

[0083] Figure 3 It is the comparison chart of defense effect of adversarial sample verification codes under three kinds of black box attacks.

[0084] Figure 4 It is the ASR of four kinds of verification codes under NC, AC and AA. DETAILED DESCRIPTION

[0085] The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0086] The anti-adversarial sample CAPTCHA attack defense method based on the application mainly includes three parts: collection and generation, character recognition and detection model construction, and generation of adversarial sample CAPTCHA, as shown in Figure 1

[0087] (1) Data collection and generation: ten real CAPTCHA pictures on the Chinese Internet were collected by web crawlers, and the collected data was processed to obtain real CAPTCHA dataset. At the same time, since the Chinese CAPTCHA has a larger classification space, only the collected CAPTCHA dataset cannot meet the training needs of the character recognition model. Therefore, a large number of simulated CAPTCHA pictures are generated for each type of CAPTCHA to expand the training set and validation set. The real CAPTCHA and the simulated CAPTCHA together constitute a complete dataset.

[0088] (2) Character detection and recognition model construction: the collected Chinese CAPTCHA is divided into 8 click types and 2 input types. For click type CAPTCHA, the attacker usually locates the character first and then recognizes it. For input type CAPTCHA, the attacker can directly recognize the character sequence with an end-to-end model, or can locate and recognize first. Therefore, the most common case is considered: character detection and recognition models are constructed respectively, and noise is generated on the characters and background of the CAPTCHA based on the two types of models.

[0089] (3) Generation of adversarial sample CAPTCHA: the application uses an adversarial sample generation method combining input transformation, gradient attack and attention mechanism, namely M-VNI-CT-FGSM, to improve the generalization of the adversarial sample and generate adversarial sample characters for the CAPTCHA character recognition model. For the threat of the CAPTCHA character detection model, the application uses an integrated adversarial sample generation method, namely SVRE-MI-FGSM, which integrates the loss functions of multiple different architecture character detection models, converts the black box setting of different architecture character detection models into a white box setting, and generates adversarial noise on the background of the CAPTCHA.

[0090] The specific steps are as follows:

[0091] Step 1: Data collection and generation.

[0092] Since there is a lack of public Chinese CAPTCHA dataset on the network at present, this embodiment selects to collect real CAPTCHA pictures from the Chinese Internet and manually label them. The collected CAPTCHA covers the current mainstream Chinese CAPTCHA. Further, since the Chinese CAPTCHA has a larger classification space, only the collected CAPTCHA dataset cannot meet the training needs of the character recognition model. New CAPTCHA datasets are generated by simulating the collected multiple CAPTCHA pictures.

[0093] 1.1. Data collection

[0094] ​(1) Data collection method

[0095] This embodiment designs an efficient, multi-threaded crawler to collect various data, which contains the following 4 parts:

[0096] Proxy pool, requestor, processor, database.

[0097] 1) Proxy pool: Since large websites set limits on the access frequency of a single IP. In order to achieve high parallel data crawling, using proxy pool can improve concurrency and crawler efficiency.

[0098] 2) Requester: The task of the requester is divided into requests for Token and requests for data. The former is to obtain a certain number of credentials required to request a large amount of data, and the latter is to obtain data using Token according to requirements.

[0099] 3) Processor: The processor is used to preprocess the original verification code pictures crawled, automatically remove duplicates, and store the processed data into the database.

[0100] 4) Database: The database saves a variety of original verification code pictures according to the source website of the verification code.

[0101] (2) Data collection strategy

[0102] This embodiment selects Shumai, Yidun, YY, Dajie, Getui, Sogou, People's Network, Baidu Cloud, 58 City and Dingxiang ten websites that provide Chinese verification code services, and obtains the verification code pictures of each website through the crawler. Among them, People's Network and Sogou use input type Chinese verification code, and others use click type Chinese verification code. All verification code samples are collected from June 2022 to July 2022. Then, data cleaning is performed to remove duplicate and damaged verification code pictures. Some verification codes are manually labeled, and the labeled information includes the position of the characters in the verification code and the character category. Then the size of the character set appearing in the collected verification code pictures is counted. All labeling work is done by four people. Discard the verification code pictures that are too complex and cannot be recognized by the labeling personnel. Each type of verification code labeled is divided into two parts, one part is used to train the verification code cracking model, and the other part is used to test the attack model and produce adversarial sample verification codes.

[0103] 1.2. Generation of expanded data set

[0104] For each kind of verification code, the application generates a large number of imitated verification code pictures to expand the training set and the verification set. In order to ensure the uniformity of the character recognition model training data, the application sets the verification code generation script to ensure that each character in the imitated data set of each kind of verification code appears the same number of times. For more complex verification code numbers and easy shields, the application adaptively generates a larger imitated data set, that is, each character appears 400 times. Considering that the character set of Sogou verification code has only 20 characters, the collected verification code is sufficient to train its cracking model, so the application no longer generates an imitated training set. Each character in the remaining all verification codes appears 200 times. Finally, the form of the original data set is that each verification code picture has a corresponding label, which records the position and character category information of the characters in the picture.

[0105] Finally, the size of the final data set established by the application is shown in Table 1.

[0106] Table 1 Size of real verification code and imitated verification code

[0107]

[0108]

[0109] Step 2: Character detection and recognition model construction.

[0110] The Chinese verification codes collected by the application are divided into 8 kinds of click type and 2 kinds of input type. For click type verification code, the attacker usually locates the character first and then recognizes it, while for input type verification code, the attacker can directly recognize the character sequence with an end-to-end model, or can locate and then recognize. Therefore, the application considers the most common case: to construct character detection and recognition models respectively. Based on the two kinds of models, noise is generated on the characters and backgrounds of the verification codes respectively.

[0111] 2.1. Character detection model construction

[0112] For the character detection model, the present application selects multiple commonly used target detection architectures: Faster-RCNN, YOLO-v5, SSD. For Faster-RCNN, the present application trains models of three kinds of backbone networks: ResNet-50, ResNet-101, VGG. For SSD, the present application trains models of two kinds of backbone networks: MobileNetV2, VGG. For YOLO-v5, the present application uses the most common CSPDarknet-53 as the backbone network for training. The label of the character detection model is set to whether there is a character, that is, the model uniformly takes the character as the foreground during training and testing, without specific character class identification. For convenience of expression, the present application defines the following'model architecture (backbone network)' to represent the target detection model, for example, Faster-RCNN(ResNet-101).

[0113] In the character detection model, Faster-RCNN(ResNet-101), SSD(MobileNetV2), YOLO-v5 are used as white box models to generate noise for the character detection model, and Faster-RCNN(ResNet-50), Faster-RCNN(VGG), SSD(VGG) and YOLO-v5 are used as black box models to test the transfer effect of the adversarial samples.

[0114] 2.2. Character recognition model construction

[0115] For the character recognition model, although the related work on the recognition model of Chinese characters is different. However, due to the complexity of Chinese verification code, these models need a deep enough network to extract features. In order to make the generation model of the present application more general, the present application selects commonly used network models with strong feature extraction capability to perform multi-classification recognition on single characters, including: Inception-Resnet-v2, Inception-v3, ResNet-50, VGG-16.

[0116] In the character recognition model, Inception-Resnet-v2 is used as a white box model to generate adversarial sample characters to defend the character recognition model, and Inception-v3, ResNet-50, VGG-16 are used as black box models to test the transfer effect of the adversarial samples.

[0117] Step 3: Generation of adversarial sample verification code.

[0118] In the adversarial sample verification code generation module, the application uses an adversarial sample generation method combining input transformation, gradient attack and attention mechanism, that is, M-VNI-CT-FGSM, to improve the generalization of the adversarial sample, and to generate an adversarial sample character for the verification code character recognition model. In the face of the threat of the verification code character detection model, the application uses an integrated adversarial sample generation method, that is, SVRE-MI-FGSM, which integrates the loss functions of multiple different architecture character detection models, converts the black box setting of different architecture character detection models into a white box setting, and generates adversarial noise on the background of the verification code. This method not only maintains the transferability of the adversarial sample between different feature extraction networks, but also realizes the transferability between multiple target detection model architectures.

[0119] For any Chinese verification code picture input into the adversarial sample generation framework, the generation framework of the application first extracts the characters according to the label. The application generates adversarial noise on each character based on the category label and the white box model, and then splices the characters to the original background without noise, and generates adversarial sample noise on the background except the characters according to the detection model and the position label of the characters.

[0120] The process of generating adversarial sample noise is opposite to the attack process. The reason is that if noise is added to the detection model first, the interference of the whole picture changes after noise is added to the recognition model, which will affect the defense effect of the adversarial sample in the detection stage.

[0121] 3.1 The application optimizes the following objectives:

[0122] Let f be the model used to generate adversarial samples, and its model parameters be θ. For any input picture x, its true label is y, and let the maximum noise amplitude at each pixel point be ε. The application needs to find an adversarial sample picture x adv , which satisfies the following conditions:

[0123] f(x adv ; θ) ≠ y (1)

[0124] ||x-x adv || ∞ <ε (2)

[0125] Further, the application maximizes the loss value of the input character picture to the true category under the model to achieve the purpose of finding an adversarial sample. Let J be the loss function for the generation model f:

[0126]

[0127] The optimization objective generates the adversarial sample under the white-box model f. The adversarial sample generation for the two stages of recognition and detection is different in input picture x, label y, and generated model f. The present application will be described in detail in the following part.

[0128] 3.2. Adversarial sample verification code generation of character recognition model

[0129] In the recognition stage, the adversarial sample generation algorithm takes a single verification code character as the input picture x, the multi-classification label of the character as the label y, and selects Inception-Resnet-v2 as the generated model f. Because Inception-Resnet-v2 is the highest accuracy in the character recognition model trained by the present application, and thanks to its fine model structure, its classification boundary can better fit the real classification boundary, so it can generate higher quality adversarial samples. The output of this stage is a single character picture with added interference.

[0130] Since the attacker's actual used character recognition model of the verification code cannot be obtained, the present application generates adversarial samples under the white-box model, and relies on the transferability of the adversarial samples to realize the defense of the black-box model. In order to be able to deceive the attacker's black-box recognition model with as little added noise as possible, the present application maximizes the transferability of the adversarial samples, and introduces three strategies to construct the adversarial sample character generation algorithm: gradient-based attack, input transformation attack, and attention mechanism.

[0131] (1) Gradient-based attack

[0132] One of the strategies to improve the transferability of adversarial samples is to improve the gradient-based optimization process. After the gradient is calculated, the input x is updated in a more general direction. The present application combines gradient variance reduction and Nesterov's fast gradient method in the gradient iteration step. Let be the adversarial sample character in the current epoch, u be the decay factor of momentum, and a be the noise amplitude added in each iteration step. Before each iteration, will first be updated to get

[0133]

[0134] When updating the gradient, the gradient variance v t and the gradient of the last step are added, and the momentum is used to update g t :

[0135]

[0136] In each epoch, after updating the gradient, the present application updates the gradient variance v according to the following formula t

[0137]

[0138] wherein

[0139]

[0140] N in formula (7) represents that the gradient of N adjacent points around the current x is sampled when calculating the gradient variance. Here, the adjacent points of x are defined as x i = x + r i , r U [-(β·ε d , (β·ε d ], wherein U [-(β·ε d , (β·ε d ] represents a random vector, each point of which is subject to a uniform distribution in the range, and β is a hyperparameter for gradient variance calculation, used to control the range of sampling. r i represents the i-th random vector; r represents a random vector sampled from U [-(β·ε d , (β·ε d ].

[0141] (2) Input transformation attack

[0142] Another strategy to improve the transferability of adversarial samples is input transformation. The idea of input transformation is similar to data augmentation, that is, to combine multiple different forms of input to make the direction of gradient update more generalized, so as to prevent overfitting of the algorithm under the current model. In the process of generating adversarial sample characters, the present application considers three input transformation strategies:

[0143] Multiple input methods: before the character picture is input into the model, it will have a certain probability p to be subjected to resize and padding to improve the generalization ability of the adversarial sample, which is represented by the present application as

[0144] D(x; p) (8)

[0145] Translation invariance method: this method considers the translation invariance of the convolutional neural network, and proposes to translate the picture to achieve the purpose of data augmentation, to improve the transferability of the adversarial sample. Further, this method uses a kernel matrix to perform convolution operation with the original picture, which is approximately equivalent to taking the average of the picture after multiple translations, which can avoid directly increasing the number of pictures and reduce the computational complexity. Let W be a predefined convolution kernel matrix. The present application represents this process as:

[0146] T(g) = W * g (9)

[0147] Scale invariance method: This method considers the scale invariance of the convolutional neural network, expands a single input picture into a series of new pictures, and takes the weighted average of the gradients of these new pictures when calculating the gradient.

[0148]

[0149] where m0 represents the number of expanded pictures, in this paper, the present application sets m0 to 4 to balance performance and speed.

[0150] (3) Attention mechanism

[0151] Although different models have different feature extraction methods, they can usually capture a large number of common features. In particular, for characters, which are simple structures, there is also a large overlap in the content that different models focus on. The present application further adjusts the noise amplitude based on the attention mechanism on the verification code characters. In order to obtain the contribution of each position in the white box model feature layer to the result, the attention of the model is approximated as the feature map The spatially pooled gradients of the final result y:

[0152]

[0153] where represents the c-th feature map of the k-th layer, and Z is the normalization factor, so that Thus, the attention weight of the c-th feature map of the k-th convolutional layer with respect to the true label y is obtained

[0154] The different feature maps are scaled by the attention weight to obtain the features that the model pays more attention to, and then the features of each channel are summed and processed using the ReLu activation function to remove negative values to obtain

[0155]

[0156] In the next step, it is added to the loss function as a regularization term, and the optimization goal is to maximize the new loss function. Unlike their method, the present application does not use the ReLu function to clip negative values when calculating , but takes the absolute value of the result after weighted summation of the feature maps, and then normalizes the result to [0, 1] again, and then reshapes the obtained to the same size as the input picture. Due to the translation invariance of the convolutional neural network, the heat map mapped back can represent the attention weight of the model at different positions in the original picture:

[0157]

[0158] Subsequently, the present application is based on An epsilon-based mask is constructed, which determines the noise amplitude that can be updated at each iteration in the iterative adversarial sample generation process:

[0159]

[0160] Wherein a = e / T, e is the maximum noise amplitude allowed at each point, T is the number of iterations, and g is a hyperparameter for amplifying the step size, which allows the position with large model attention weight to be updated with a larger step size each time. Note that although each pixel point can be updated with a larger amplitude each time, the present application still clips the picture at the end of the iteration to ensure that ||x-x adv || ∞ < e.

[0161] There are two reasons for the above changes: first, in the feature map If the value of a certain position is positive, it means that it has a positive impact on the ground true class y, i.e. the point makes the probability of class y in the prediction output larger. If it is negative, it means that the pixel point has a negative impact on class y, i.e. the point makes the probability of class y in the prediction output smaller. The greater the absolute value of the point, the greater the impact on the result. The present application believes that in the context of generating adversarial samples, directly setting the points in the negative part to 0 will reduce the effect of attention mechanism. Therefore, the present application sets Take the absolute value, i.e. the greater the absolute value of the point, the greater the update amplitude, and the direction of the update is determined by the present application when running the algorithm based on FGSM, i.e. sign(g), so that the information in the attention weight is fully utilized, whether it is positive or negative to y. Secondly, the heat map is mapped back to the original picture rather than optimized based on the regularization term, because The optimization direction information in the gradient sign method cannot be effectively utilized in the regularization term, and the loss function needs to be used multiple times in the calculation process in equation (7). Optimizing the loss function based on attention to calculate the regularization term each time will slow down the running speed of the algorithm. Therefore, the present application chooses to reshape it back to the size of the input after

[0162] (4) Generation process of adversarial samples for recognition model

[0163] ​Based on the above method, the generation process of the recognition model against the adversarial sample is proposed, which is an iterative method based on gradient flag. The above three defense strategies are combined together, the update amplitude (M) of each point is determined by the attention mask, the area commonly focused by the model is destroyed, in each iteration process, the input is comprehensively transformed (CT) to enhance the data and improve the generalization ability of the algorithm, and the gradient variance reduction (V) and Nesterov accelerated gradient method (NI) are combined when the gradient is updated to prevent the algorithm from falling into local optimum. In the process of calculating the gradient, the present application simply uses the class cross-entropy J as the loss function. The algorithm of the present application is named M-VNI-CT-FGSM, and its process is shown in the algorithm as shown in Figure 2

[0164] In the actual generation process, the present application cuts out all single characters in the verification code according to the pre-labeled character position label, and sends them into the model together with their class labels, adds adversarial noise, and then pastes them back to the original position.

[0165] 3.3. Adversarial sample generation of character detection model

[0166] In the adversarial sample generation algorithm in the detection stage, the present application takes the whole verification code picture as the input x, and its corresponding label y is defined as the frame representing the character position and the classification label at the position. It is noted that in the detection stage, the label of the present application only has foreground (character) and background (not character). The present application selects three target detection models, Faster-RCNN (ResNet101), YOLO-v5 and SSD (MobileNetV2) as the generation model, and uses an integrated attack method to generate adversarial samples against the detection model.

[0167] On the adversarial sample of the character detection model, the present application uses an adversarial sample generation method integrating multiple model loss functions. This method is based on an advanced integrated adversarial sample generation algorithm. This algorithm performs integrated attack based on random gradient variance reduction on multiple recognition models, attacks the integrated model under the consideration of gradient variance, and finds the local minimum of multiple models, so that the algorithm can be more easily migrated to other black box models. Different from this algorithm, the present application uses multiple character detection models with different architectures as integrated models to generate adversarial samples. Because for detection models with different architectures, the path to find the local maximum will have a huge deviation, resulting in that the adversarial samples generated by any one of the architectures cannot be well migrated to other architectures. Therefore, the present application uses this method to find a gradient variance smaller optimization path in multiple architecture detection models based on integrating multiple models and considering gradient variance reduction.

[0168] ​The application redefines the loss function, considers three models participating in the integrated attack, Faster-RCNN, YOLO-v5 and SSD. The loss function used when the application directly uses these models for training is used as the loss function for generating adversarial samples.

[0169] L frcnn =L rpnCls +L rpnLoc +L roiCls +L roiLoc (15)

[0170] Wherein, L rpnCls represents the classification loss of the RPN module of Faster-RCNN, L rpnLoc represents the bounding box regression loss of the RPN module of Faster-RCNN, which is used to train the positioning ability of the model. L roiCls represents the ROI module classification loss, L roiLoc represents the bounding box regression loss of the ROI module.

[0171] L yolov5 =L loc +L cls +L conf (16)

[0172] Wherein, L loc is the position loss of YOLO-v5, L cls is the classification loss of whether to contain an object. L conf is the multi-class confidence loss.

[0173] L ssd =L loc ′+L cls ′ (17)

[0174] Wherein, L loc ′ is the position loss of ssd, L cls ′ is the classification loss.

[0175] In the process of integrated attack, the application weights and averages these three losses based on different weights w to obtain an integrated loss:

[0176] J ens (x,y;θ)=w1L frcnn (x,y;θ1)+w2L yolov5 (x,y;θ2)+w3L ssd (x,y;θ3) (18)

[0177] where θ1, θ2, θ3 represent the model parameters of Faster-RCNN, YOLO-v5 and SSD respectively, and w1, w2, w3 represent the weights of the three loss functions respectively.

[0178] Based on the SVRE-MI-FGSM algorithm for generating adversarial samples, the loss function is modified to the J ens (x,y; θ), in each round of inner loop, the algorithm randomly selects one from L frcnn , L yolov5 and L ssd to reduce the gradient variance and update the gradient.

[0179] In the actual generation process, the picture with characters added with adversarial noise is input into the algorithm, and a mask with the same size as the input picture is set at each iteration, the value of the position containing the character is 0, and the value of the position not containing the character is the maximum noise amplitude ε, that is, the noise at the character is not updated to prevent mutual interference. Finally, after the adversarial attack on the detection model, the adversarial sample verification code of the application can deceive various character recognition models and character detection models.

[0180] Experiment:

[0181] 4.1. Experimental setup

[0182] The application designs three experiments to evaluate the defense effect of adversarial sample verification code. In the experiment, all model training and adversarial sample generation are performed on four servers with 43G memory Intel(R) Xeon(R) Platinum 8255C CPU and 24GB video memory NVIDIA GeForce RTX-3090.

[0183] Three evaluation indexes for measuring the attack ability of the model on the verification code are defined:

[0184] Detection success rate (DSR) = number of verification code images in which all characters are located / total number of verification code images;

[0185] Recognition success rate (RSR) = number of verification code images in which all characters are correctly recognized / total number of verification code images;

[0186] Attack success rate (ASR) = number of verification code images in which the attack is successful / total number of verification code images.

[0187] It is to be noted that for the click-type Chinese CAPTCHAs and the input-type Chinese CAPTCHAs, the definition of successful attack is different according to their different characteristics. For the click-type CAPTCHAs, if the attack model can correctly give all the characters contained in the CAPTCHA label, it is considered as a successful attack; for the input-type CAPTCHAs, because the user is required to input all the characters in the CAPTCHA, any additional answer is wrong, therefore, if the output characters of an attack model are exactly the same as the characters in the CAPTCHA label, it is considered as a successful attack. The definition of successful detection is that for each box (rectangle covering the character) in the CAPTCHA label, there is a box b' in the prediction result, which satisfies their intersection union IOU b,b' < 0.5.

[0188] At the same time, the Top-1 accuracy is used to evaluate the degree of influence of the adversarial perturbation on the recognition of characters. In order to further study the defense effect of the adversarial perturbation on the detection model, the mean number of mistakes (MMN) is defined. It refers to the average number of predicted boxes that do not contain the characters in the label for each CAPTCHA. A higher MMN means that the misleading effect of the adversarial example on the detection model is better.

[0189] The maximum perturbation ε is set to 0.1, which means that the change of each pixel in the CAPTCHA is less than 10%. In M-VNI-CT-FGSM, the maximum perturbation γ added by the attention mask in formula (5) is set to 0.03, and the loss function introduced in formula (6) is used in SVRE-MI-FGSM, and w1 = w2 = w3 = 1 is set in the three white-box models f1, f2, f3

[0190] 4.2. Defense capability experiment of adversarial sample large character set CAPTCHA

[0191] In this experiment, the effectiveness of the adversarial sample CAPTCHA against the black-box attack is demonstrated. For each of the eight CAPTCHAs, 1000 clean CAPTCHAs in the test set are used to generate adversarial sample CAPTCHAs. Then, the clean CAPTCHAs and the adversarial sample CAPTCHAs are input into three black-box attacks:

[0192] (1) SR: that is, using SSD (VGG) to detect characters and using ResNet-50 as a character recognition model;

[0193] (2) FA: that is, using Faster-RCNN (ResNet-50) to detect characters and using an attention-based model to recognize characters. For the input-type CAPTCHAs of people.com.cn and sogou, the characters are directly recognized using an end-to-end attention-based model. In this case, the recognition and detection processes cannot be separated, so only their ASR indicators are measured;

[0194] (3) EM: i.e. end-to-end model with Mask-RCNN architecture. In this case, recognition and detection cannot be separated either, only the ASR metric is measured.

[0195] The experimental results are shown in Table 2. It can be seen that, first of all, the DSR, RSR and ASR of the clean CAPTCHAs are relatively high, especially the value of ASR, which is from 6.5% to 100.0%, which proves that the commonly used attack model is enough to crack the CAPTCHAs of large character sets. Secondly, the three indicators of the adversarial sample CAPTCHAs can be greatly reduced. The average ASR of clean CAPTCHAs and the average ASR of adversarial CAPTCHAs are calculated, and the results show that after adding adversarial perturbations, the average ASR decreases from 53.33% to

[0196] 3.49%, especially the end-to-end model and the attention mechanism-based model which are quite different from the white-box model can also be successfully defended. In addition, the RSR of the eight CAPTCHAs also greatly reduces to close to 0.0%. The DSR of most schemes also decreases. However, the DSR of the Dyzan and Sogou does not decrease significantly, and the reasons are analyzed as follows: for the Dyzan CAPTCHA, the difference between the characters and the background is quite obvious, so it is difficult to make the detection model "believe" that the characters in the perturbed CAPTCHA are background areas; for Sogou, the character position of this input-type CAPTCHA is quite fixed, and the model is easy to learn this feature, so it is more difficult to mislead the attack model. Thirdly, as seen in Figure 3 , the perturbation in the adversarial CAPTCHA is not obvious, which means that the usability of these CAPTCHAs is maintained.

[0197] Table 2 Transferability evaluation of four CAPTCHAs against character recognition model attacks

[0198]

[0199] In summary, the generation method of the adversarial CAPTCHA of the present application can reduce the success rate of black-box model attacks while maintaining the usability of the CAPTCHA.

[0200] 4.3. Transferability comparison experiment of adversarial sample CAPTCHA

[0201] This experiment selects four representative CAPTCHAs, which are: Dyzan (multiple fonts), Yidun (character distortion, complex background), YY (hollow characters, multiple fonts), and Dajie (character tilt, thin font). The selected white-box model is used to generate adversarial characters and adversarial CAPTCHAs, and then multiple black-box models are used to attack them to test their transferability.

[0202] (1) Transferability against character recognition attacks

[0203] For each type of character, 1,000 characters were randomly selected in the test set to generate adversarial characters. Then, these characters were attacked using a white-box model and five black-box models. They are Inception-Resnet-v2, Inception-v3, ResNet-50, VGG-16, an ensemble model that averages the prediction probabilities of Inception-v3, ResNet-50, VGG-16, and an attention-based model, which are denoted as R1, R2, R3, R4, R5 and R6, respectively. Five baseline methods were used to compare the transferability. They are VNI-CT-FGSM, NI-FGSM, FGSM and DeepFool. VNI-CT-FGSM is the ablation result of the algorithm steps of the present method without attention mechanism, VNI-FGSM is the ablation method that cancels the input transformation, FSGM is the most basic fast gradient sign attack, and DeepFool is another classic method of generating adversarial samples. At the same time, clean characters were used for comparison, and random Gaussian noise with the same perturbation amplitude ε was added to the characters.

[0204] The final results are shown in Table 2. First, all character recognition models achieved high accuracy on clean characters and characters with random noise. Among them, Inception-Resnet-v2 has the highest character recognition accuracy, while VGG-16 has the lowest. Second, the white-box model Inception-Resnet-v2 using the adversarial characters generated by the present method can significantly reduce the accuracy of multiple white-box and black-box models. Third, gradient-based attacks, input transformations, and attention mechanisms can all improve the transferability of adversarial characters. After integrating these methods, the present method achieved the best defense effect.

[0205] (2) Transferability against character detection attacks

[0206] For each type of CAPTCHA, 1,000 CAPTCHAs in the test set were used to generate adversarial CAPTCHAs. First, perturbations were added on the characters, and then perturbations were added on the background. Three white-box models and three black-box models were used to attack these adversarial CAPTCHAs, the white-box models are Faster-RCNN (ResNet-101), SSD (MobileNetV2) and YOLO-v5, denoted as D1, D2 and D3, respectively; the black-box models are Faster-RCNN (ResNet-50), Faster-RCNN (VGG), SSD (VGG), denoted as D4, D5 and D6, respectively. Clean CAPTCHAs were used for comparison, and three kinds of perturbations were added, random Gaussian noise, perturbations generated by ENS-MI-FGSM (random variance reduction) and noise generated by SVRE-MI-FGSM. Both ENS-MI-FGSM and SVRE-MI-FGSM use the loss function in (6). Then the test set with perturbations was input into the white-box and black-box detection models and the DSR and MMN of these detection models were evaluated.

[0207] The experimental results are shown in Table 3. First, all character detection models have achieved high DSR and low MMN on CAPTCHA images without noise and random noise. This means that the character detection models rarely make mistakes on CAPTCHAs without adversarial perturbations; second, detection models with multiple architectures and feature extraction modules have lower DSR and higher MMN when attacking adversarial CAPTCHAs, lower DSR means that the defense measures of the present application greatly reduce the ability of black-box and white-box models to detect characters, so only a small number of adversarial CAPTCHAs have the positions of all characters, higher MMN means that adversarial samples make the detection model give more predicted positions, that is, they mistake part of the background for characters. These results show that considering the ensemble method of different object detection structures can generate adversarial CAPTCHAs with transferable defense capabilities against detection attack models; finally, compared with random noise and ENS-MI-GSM, using SVRE-MI-GSM can further improve the defense capability against character detection models.

[0208] Table 3 Transferability evaluation of four types of CAPTCHAs attacking character detection models

[0209]

[0210]

[0211] 4.3. Robustness experiment of adversarial sample CAPTCHAs

[0212] In real-world scenarios, once attackers discover the existence of adversarial CAPTCHAs, they can employ more sophisticated methods to counter these adversarial examples and crack the CAPTCHAs. This experiment selects one of the most effective attack methods: using adversarial examples to train the model adversarially. It is assumed that attackers can collect a limited number of adversarial CAPTCHAs to fine-tune their attack model trained on a normal CAPTCHA dataset. Using CAPTCHAs from YiDun, Dajie, YY, and GeeTest as examples, adversarial perturbations of 1,000 CAPTCHAs are generated from the training set. Then, characters are extracted from all the perturbated CAPTCHAs to fine-tune the ResNet-50 character recognition model trained on normal characters. Next, these perturbated CAPTCHAs are used to fine-tune the SSD (VGG) character detection model trained on normal CAPTCHAs.

[0213] The ASR index was measured and compared under three different conditions:

[0214] 1)NC: Using SSD (VGG) and ResNet-50 for normal training to attack clean CAPTCHAs.

[0215] 2) AC: Use SSD (VGG) and ResNet-50 to fine-tune through adversarial training to attack clean CAPTCHAs.

[0216] 3) AA: Fine-tuning with SSD (VGG) and ResNet-50 through adversarial training to attack adversarial CAPTCHAs.

[0217] The results are as follows Figure 4 As shown, after fine-tuning the model using adversarial CAPTCHAs, the ASR (Adversarial Success Rate) of the attack model increased. Taking GeeTest as an example, its ASR increased from 1.2% (as shown in Table 2) to 37.8%. However, AA's ASR of 37.8% is still significantly lower than NC's 93.9% and AC's 93.5%, meaning that the adversarial CAPTCHAs generated in this invention still offer much better defense than CAPTCHAs without adversarial perturbations. Therefore, even if attackers obtain adversarial CAPTCHAs to fine-tune their models, the adversarial CAPTCHAs generated in this invention can still maintain strong defense against the fine-tuned models.

[0218] To sum up, the application provides a kind of big character set verification code attack defense method based on adversarial sample, and the defense based on adversarial sample is applied based on Chinese verification code of 10 popular websites, and various black box models are used, including the common method for cracking verification code to attack it.The experimental results show that the generated adversarial sample makes the detection and recognition success rate of the character of the attack model of the various black boxes decrease significantly, indicating that the adversarial sample verification code has good transferability.At the same time, the adversarial sample verification code can resist the scene of using filter and adversarial training by the attacker to a certain extent, showing that the adversarial sample verification code has good robustness.The application achieves excellent results in generating adversarial sample verification code to resist verification code attack.

Claims

1. An attack defense method for large character set Captcha based on an adversarial sample, characterized in that, The method comprises the following steps: Step 1: Collect existing large character set verification code data on a Chinese verification code service website and manually label them, and generate new verification codes by imitation to build an expanded verification code data set; Step 2: Build a character detection model and a character recognition model, the character detection model includes a white box model for generating noise for the character detection model, and a black box model for testing the migration effect of the adversarial sample; the character recognition model includes a white box model for generating adversarial sample characters to defend against the character recognition model, and a black box model for testing the migration effect of the adversarial sample; noise is generated on the characters and background of the verification code based on the character detection model and the character recognition model respectively; Step 3: Generation of adversarial sample verification code Step 3.1: In the process of generating adversarial sample verification code, an adversarial sample generation method combining gradient attack, input transformation and attention mechanism is used to generate adversarial sample characters against the character recognition model of the verification code; Step 3.2: In the face of the threat of the character detection model of the verification code, an integrated adversarial sample generation method is used to integrate the loss functions of multiple different architecture character detection models, convert the black box settings of different architecture character detection models into white box settings, and generate adversarial noise on the background of the verification code to defend against the character detection model; The step 3 of generating adversarial sample characters against the character recognition model of the verification code specifically comprises: Step 3.1.1: Determining a loss function for the adversarial sample generation model Step 3.1.1: Determining a loss function for the adversarial sample generation model Picture of adversarial sample sought Satisfies the following conditions （1） （2） wherein, a parameter of the adversarial sample generation model, an arbitrary input character picture, a class true label of the picture, a maximum noise amplitude at each pixel point;​​ By maximizing the adversarial sample generation model Lower input character picture The loss value of the real category is used to achieve the purpose of finding the adversarial sample, and the category cross entropy is adopted As an adversarial sample generation model Loss function: （3） Step 3.1.2: Gradient-based attack; The calculation obtains The derivative of the loss function In the gradient iteration step, gradient variance reduction and Nesterov iteration are combined; before each iteration, the adversarial sample character in the current epoch is An update is performed to obtain : （4） wherein, is the adversarial sample character in the current epoch, is the momentum decay factor, is the noise amplitude added in each step of iteration, is the updated adversarial sample character; is the gradient of the iteration. In each epoch, after updating the gradient, the gradient variance is updated according to the following formula : （5） wherein, is the updated gradient variance; and （6） In the formula, This represents the value of the current gradient variance when calculating the gradient variance. Surrounding The gradient of each nearest point is sampled; The nearest point is defined as ,in This represents a random vector, where each point follows a uniform distribution within a range. The hyperparameters for calculating gradient variance are... Representing the 1 random vector; Representative sampling from A random vector; Indicates before random sampling With respect to the derivative of the loss function, express The derivative of the loss function with respect to the nearest point; During gradient update, the gradient variance obtained in the previous step is used. and The derivative of the loss function Add, based on momentum update of the first gradient of the next iteration : （7） wherein, is the momentum updated gradient, denotes the LI norm; Step 3.1.3: Input transformation attack Adopting multiple input methods: before the character picture is input to the adversarial sample generation model, with probability Applying shaping and padding to it: （8） Translation invariance method: translate the picture to achieve the purpose of data augmentation, and use the kernel matrix to convolve with the original picture: （9） wherein, is a predefined convolution kernel matrix, is a gradient after a convolution operation, is an original gradient; Scaling invariance method: expand a single input picture into a series of new pictures, and take the weighted average of the gradients of these new pictures when calculating the gradient: （10） wherein, represents the number of pictures after the amplification; Step 3.1.4: Adjust the noise amplitude on the verification code characters based on the attention mechanism To obtain the contribution of each position in the white-box model feature layer to the result, the attention of the adversarial sample generation model is approximated as a feature map of the input picture the real label of the spatial pooling gradient: （11） wherein, represent the first layer of the first feature map of the first convolutional layer, is a normalization factor such that ; the attention weight of the first feature map of the first convolutional layer is obtained ; represents a white-box model; , n represent the horizontal and vertical coordinates of the pixel point, respectively; Different feature maps are passed through attention weights Downscaling is performed to obtain features to which the model pays more attention. Subsequently, the features of each channel are summed and processed using a ReLu activation function to remove negative values to obtain: （12） wherein, denotes the attention activation map with negative values removed; The result of the weighted sum of the feature maps is taken as an absolute value, and the result is normalized to [0, 1] again, and then the obtained is reshaped to the same size as the input picture; due to the translation invariance of the convolutional neural network, the heat map mapped back shows the attention weight of the model at different positions in the original picture: （13） wherein, denotes the attention activation map after taking the absolute value; According to A mask is constructed based on the magnitude of noise that can be updated at each iteration in the iterative adversarial sample generation process. （14） where, is the noise update amplitude for each epoch, , is the maximum noise amplitude allowed at each point, is the number of iteration rounds, is a hyperparameter for the amplification step, which crops the image at the end of the iteration to ensure .

2. The attack defense method against large character set verification codes based on adversarial samples according to claim 1, characterized in that, In step 1, the annotation information of the verification code data and the manual annotation includes: the position and character category of the characters in the verification code; first, count the size of the character set appearing in the collected verification code pictures; then, divide each annotated verification code into two parts, one part is used to train the verification code cracking model, and the other part is used to test the attack model and produce adversarial sample verification code.

3. The method of claim 1, wherein the method is characterized by, In step 2, the construction of the character detection model and the character recognition model further comprises: 1) Select Faster-RCNN, YOLO-v5 and SSD as the target detection architecture for the character detection model; for Faster-RCNN, train models of three kinds of backbone networks of ResNet-50, ResNet-101 and VGG; for SSD, train models of two kinds of backbone networks of MobileNetV2 and VGG; for YOLO-v5, use CSPDarknet-53 as the backbone network for training; The label of the character detection model is set as whether there is a character, that is, the character detection model uniformly takes the character as the foreground during training and testing, without identifying the specific character category; The Faster-RCNN using ResNet-101 as the backbone network, the SSD using MobileNetV2 as the backbone network, and the YOLO-v5 as the white box model; the Faster-RCNN using ResNet-50 and VGG as the backbone network, and the SSD using VGG as the backbone network as the black box model; 2) The character recognition model selects the network model Inception-Resnet-v2, Inception-v3, ResNet-50 and VGG-16 to perform multi-classification recognition on a single character; Inception-Resnet-v2 is used as a white box model, and Inception-v3, ResNet-50 and VGG-16 are used as black box models.

4. The method of claim 1, wherein the method is characterized by, Step 3.2 specifically includes: The loss function used in the training of the three models participating in the integrated attack, Faster-RCNN, Yolo-v5 and SSD, is used as the loss function for generating the adversarial samples for the detection model; Loss function of faster-rcnn when training is: （15） wherein, represents the classification loss of the RPN module of Faster-RCNN, represents the Bounding Box regression loss of the RPN module of Faster-RCNN, for training the localization ability of the model; represents the ROI module classification loss, represents the Bounding Box regression loss of the ROI module; Loss function for YOLO-v5 training is: （16） wherein, is a position loss of YOLO-v5, is a classification loss of whether to contain an object, is a multi-classification confidence loss; Loss function when training the SSD is: （17） wherein, is a position loss of the SSD, is a classification loss; In the process of integrated attack, the three losses are weighted and averaged based on different weights to obtain an integrated loss: （18） wherein, , , respectively represent the model parameters of Faster-RCNN, YOLO-v5 and SSD; , , respectively represent the weights of three loss functions.

Images