Image semantic adversarial attack sample generation system and generation method based on image brightness mapping

Abstract

The present invention discloses an image semantic adversarial attack sample generation system based on image brightness mapping, including: an image reading module, used to read the original image information, separate the color and brightness of the image, and obtain the image brightness matrix and the image color matrix; an image mapping module, used to intercept the effective brightness matrix and the invalid part of the image according to the brightness matrix of the original image, and use the mapping function to map the effective brightness matrix to obtain the original attack sample; a target network module, used to put the original attack sample into the target network for reasoning, and back propagate after obtaining the reasoning result, adjust the mapping function, and obtain a new mapping function; an image evaluation module, used to select to skip, save or update the current original attack sample according to the image reasoning result and the image quality score. The present invention also discloses a method for generating attack samples using the above-mentioned attack sample generation system.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security and relates to an image semantic adversarial attack sample generation system and generation method based on image brightness mapping. Background Art

[0002] Deep neural networks (DNNs) are vulnerable to attacks from targeted adversarial examples. p Some attacks do not restrict the perturbations to achieve better robustness and transferability, so these adversarial examples can be used for transfer-based black-box attacks. But these methods discard a large part of naturalness. Summary of the invention

[0003] In order to solve the shortcomings of the prior art, the purpose of the present invention is to propose an image semantics anti-attack system based on image brightness mapping, which can be used without restrictions (such as L p norm) while maintaining naturalness; by generating natural attack images for the target network based on the input RGB image, the classifier can make mistakes when it is unrecognizable to the human eye.

[0004] Specifically, the present invention proposes an image semantic adversarial attack sample generation system based on image brightness mapping, comprising:

[0005] An image reading module is used to read the original image information, separate the color and brightness of the original image, and obtain an image brightness matrix and an image color matrix;

[0006] Preferably, the input image format of the image reading module is Tensor, and before the brightness and color of the original image are separated, it is converted from the RGB space to a LAB space image to obtain a Tensor of the LAB color space.

[0007] An image mapping module is used to intercept the effective brightness matrix and the invalid part of the image according to the brightness matrix of the original image, and use a mapping function to map the effective brightness matrix. After the mapping is completed, it is integrated with the invalid part of the image to obtain the original attack sample;

[0008] Preferably, the image mapping module comprises:

[0009] The mapping division unit is used to divide the effective brightness matrix based on the effective brightness value of the image and the invalid part of the image according to the brightness range of the original image; in particular, the effective brightness range of the image is the minimum value to the maximum value of the image in the brightness space.

[0010] An image mapping unit is used to initialize a mapping function and map the effective brightness matrix according to the mapping function, and integrate the mapped effective brightness matrix with the invalid part of the image obtained in the mapping division unit to obtain an original attack sample;

[0011] The target network module is used to put the original attack sample into the target network for inference, and after obtaining the inference result, back-propagate and adjust the mapping function to obtain a new mapping function;

[0012] Preferably, the target network module includes:

[0013] Image reasoning unit: puts the original attack sample into the image reasoning unit for reasoning and obtains the reasoning result. If it is an image classification task, it classifies the image and determines the category to which the image belongs.

[0014] The regularization unit is used to make the loss value contain a term whose slope of the mapping function is not zero.

[0015] Preferably, the loss function after adding the regularization term is:

[0016] Loss = L C&W (x,l)-ηsum(|θ|) / K

[0017] Among them, L C&W (·) is the C&W loss, that is, the value of the class corresponding to label l in the inference result obtained by the image inference unit after the input image x is obtained minus the maximum value outside the class, sum(|θ|) / K is the regularization term; η is the regularization rate, which is used to adjust the degree of regularization, that is, the proportion of the regularization term in the loss function. In the present invention, η=0.5.

[0018] The back propagation unit is used to modify the mapping function according to the obtained inference result and the value of the regularization term obtained by the regularization unit and in combination with the learning rate; that is, according to the inference result, the regularization term, and the network model, the partial derivative value of the mapping function to the inference result is obtained and the mapping function is adjusted in combination with the learning rate to obtain a new mapping function;

[0019] Preferably, the learning rate is determined by experimental results; the learning rate used in the present invention is 0.3;

[0020] Preferably, the direction of parameter modification, i.e., whether to increase or decrease the parameter value, is determined according to the partial derivative value, i.e., the degree of influence of the parameter of the mapping function on the final result; in particular, the partial derivative value can be directly calculated;

[0021] Preferably, after the direction of parameter adjustment is determined by the partial derivative, the size of each step of adjustment is the learning rate; in particular, if the parameter θ is obtained after a certain calculation k The partial derivative is positive, and the learning rate is 0.3. Then the parameter θ after this iterationk Add 0.3 to the value to adjust;

[0022] The image evaluation module is used to decide whether to skip, save or update the current image based on the image reasoning result and the image quality score.

[0023] Preferably, the image evaluation module comprises:

[0024] An image quality evaluation unit, which obtains an image quality score of the generated original attack sample through an image evaluation algorithm;

[0025] The image decision unit skips, saves or updates the current image according to the inference result obtained by the image inference unit and the image quality score obtained by the image quality assessment unit.

[0026] In the image mapping module, preferably, each slope segment represented by the mapping function is randomly initialized within the brightness range of the original image.

[0027] Preferably, the mapping function is randomly initialized, that is, the mapping function uses random values, which can better exert the ability of the attack system than the mapping function that is initially all 1.0, and obtain a higher attack success rate and image quality;

[0028] Preferably, in the present invention, the parameter value of the mapping function is set to a random floating point number between 0.8 and 1.2;

[0029] Preferably, the image mapping unit divides the brightness into segments evenly according to the mapping function, and the brightness in each segment changes in value with the same slope according to the corresponding parameter.

[0030] In the target network module, preferably, the image reasoning unit is a convolutional neural network, including but not limited to: LeNet, ResNet, DenseNet.

[0031] Preferably, the regularization term is the inverse of the average of the absolute values ​​of the mapping function values.

[0032] In the image evaluation module, preferably, the image quality evaluation unit uses LPIPS as a scoring standard to obtain an image quality score.

[0033] Preferably, LPIPS is an evaluation index that imitates human visual perception and is used to measure the similarity of images in a manner consistent with human judgment.

[0034] Preferably, the lower the LPIPS value, the better the image quality, and the higher the image quality score, which is the inverse of the LPIPS value, is.

[0035] Preferably, the image decision unit compares the image quality score obtained by the image quality assessment unit for the successfully attacked image with the image quality score of the attack sample saved in the previous iteration, and selects the image with the higher quality score to save; if the current image attack is unsuccessful, it is saved as a failed image. In the image classification task, the successful attack means that the category and label of the image after the attack are different from the image inferred by the image reasoning module; the unsuccessful attack means that the category and label of the image after the attack are the same as the image inferred by the image reasoning module.

[0036] The present invention also proposes a method for generating an image semantic adversarial attack image based on image brightness mapping, the method comprising the following steps:

[0037] Step 1: Input the original image into the image reading module, then convert the original image into Tensor format, and convert it from RGB color space to LAB color space to obtain the brightness matrix and color matrix of the input image;

[0038] Step 2: The image mapping module extracts the effective brightness matrix from the original image brightness matrix obtained in step 1, and maps the effective brightness matrix using a mapping function. After the mapping is completed, it is integrated with the invalid part of the image to obtain the original attack sample;

[0039] Step 3: Input the original attack sample into the target network module, use the image inference unit to perform inference and obtain the inference result, and the back propagation unit corrects the mapping function according to the classification result obtained by inference and the value of the regularization term obtained by the regularization unit and the learning rate;

[0040] Step 4: The image evaluation module selects to skip, save or update the original attack sample obtained in step 2 by comparing the inference result in step 3 and the image quality score obtained by the image quality evaluation module;

[0041] Step 5: Repeat steps 2 to 4 to obtain the final attack sample.

[0042] In step 1, the image reading module reads the image through the OpenCV library of python, adjusts the image format from BGR to RGB using the Numpy library, and then obtains the brightness information and color information of the image through the conversion formula between RGB space and LAB space; the original image is converted into Tensor format through the torch library of python; the formula for converting the data in Tensor format from RGB color space to LAB color space is as follows:

[0043]

[0044]

[0045]

[0046]

[0047]

[0048] Among them, R, G, B represent the three channel components of the original image in the RGB color space, and they are all two-dimensional matrices with the same size as the image; X, Y, Z are the three channel components represented in the XYZ color space obtained by representing the image in the RGB color space; X n ,Y n ,Z n is a fixed value, usually X n =0.95047,Y n =1.0,Z n =1.08883; Through matrix operations on the XYZ color space, the representation of the original image in the LAB color space can be obtained.

[0049] In step 2, the setting of the effective brightness is determined according to the maximum and minimum values ​​of the input image in the brightness space; that is, in the brightness space, the part of the matrix value greater than 0 after subtracting the minimum value in the brightness matrix is ​​the effective brightness of the input image; after the effective brightness matrix is ​​intercepted, the invalid part of the image is saved at the same time;

[0050] The mapping function is as follows:

[0051]

[0052]

[0053] Among them, K represents the number of segments, θ k represents the parameter corresponding to the kth segment, x k Represents the brightness value of any position in the original image, F θ (x k ) is the new brightness value of the corresponding position of the image after mapping.

[0054] In step three, the image inference unit is a convolutional neural network, which usually accepts input (b, 3, W, H) and extracts features through several convolutional layers and several pooling layers, and then connects to a fully connected layer with a classification number c for classification, where b is the batch size, 3 represents RGB three dimensions, W is the image width, and H is the image height; in the image classification task, after one or b pictures that have been read into the Tensor format are input into the target network, a 1-dimensional vector representing the category to which the target network judges the picture belongs is obtained. The number of categories of the 1-dimensional vector is c parameters, which respectively represent the possibility of the category to which the target network infers the picture, and the category with the highest value is the category of the picture inferred by the target network;

[0055] Specifically, the network structure of LeNet is as follows Figure 5 As shown in the figure, for the input image, after several convolutional layers, such as C1, C3, and several pooling layers (downsampling layers), such as S2, S4, after extracting features, it is connected to several fully connected layers (usually one) as shown in the figure above, C5, F6, OUTPUT, where the number of OUTPUT layer parameters is the number of target categories, such as Figure 5 For handwritten digit recognition, there are 10 digits from 0 to 9, so the OUTPUT layer has 10 parameters; the remaining ResNet and DenseNet have similar network features;

[0056] In step 4, for the attack image generated in a certain iteration, we put it into the image reasoning module in step 3 to check whether the generated image can cause the image reasoning module to misclassify it. If it is misclassified, it means the attack is successful. For the image with successful attack, its score in the image evaluation module is compared with the current saved image, that is, the corresponding score of the attack sample saved in the previous iteration. If the score is higher, the new image is saved, otherwise the new image is discarded and the next iteration is entered.

[0057] The beneficial effects of the present invention include: a groundbreaking attack on image brightness; based on the characteristics of brightness, an innovative targeted regular optimization is proposed, and under this optimization, the accuracy of the attack and the image quality are significantly improved; in a variety of image-related tasks and multiple image data sets, the attack of the present invention has achieved excellent attack success rates, and is significantly better than the existing unrestricted attack methods in multiple commonly used image quality evaluation indicators. Through the image brightness attack sample generation system based on the derivable mapping of the present invention, it is possible to obtain attack samples with excellent image quality with a high attack success rate, so that the image looks natural to the human eye, while making the target model classification error. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of an image semantic counterattack system based on image brightness mapping provided by the present invention;

[0059] Figure 2 It is a structural schematic diagram of an image semantic counterattack system based on image brightness mapping provided by the present invention;

[0060] Figure 3 This is a mapping example diagram of an image semantics counterattack attack system based on image brightness mapping provided by the present invention;

[0061] Figure 4 It is a flowchart of a method for generating image semantic adversarial attack samples based on image brightness mapping provided by the present invention.

[0062] Figure 5 Schematic diagram of the convolutional neural network structure in the image reasoning unit 1 of the present invention.

[0063] Figure 6 Schematic diagram of the original image, the image generated by the existing PGD method, and the image generated by the OLF method of the present invention in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The invention is further described in detail with reference to the following specific embodiments and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0065] According to the implementation of the present application, a system for generating image semantic adversarial attack samples based on image brightness mapping is proposed, such as Figure 1 As shown, including:

[0066] The image reading module is used to convert the input original RGB image into a LAB space image and separate the color and brightness of the image to obtain the image brightness matrix and the image color matrix;

[0067] An image mapping module is used to intercept the effective brightness matrix and the invalid part of the image according to the brightness matrix of the original image, and use a mapping function to map the effective brightness matrix. After the mapping is completed, it is integrated with the invalid part of the image to obtain the original attack sample;

[0068] The target network module is used to put the original attack sample into the target network for inference, and after obtaining the inference result, back-propagate and adjust the mapping function to obtain a new mapping function;

[0069] The image evaluation module is used to decide whether to skip, save or update the current image according to the image reasoning result and the image quality evaluation result.

[0070] like Figure 2 As shown, the image mapping module includes:

[0071] A mapping division unit, used for dividing the effective brightness matrix based on the effective brightness value of the image and the invalid part of the image according to the brightness range of the original image;

[0072] The image mapping unit is used to initialize the mapping function. In the present invention, the parameter value of the mapping function is set to a random floating point number of -0.2 to 0.8, which can better exert the ability of the attack system than the mapping function with the initial value of all 1.0, and obtain a higher attack success rate and image quality. The effective brightness matrix is ​​mapped according to the mapping function, and the mapped effective brightness matrix and the invalid part of the image obtained in the segmentation unit are integrated to obtain the original attack sample.

[0073] like Figure 2 As shown, the target network module includes:

[0074] The image reasoning unit puts the original attack sample into the target network for reasoning and obtains the reasoning result;

[0075] The regularization unit is used to make the loss value contain a term whose slope of the mapping function is not zero.

[0076] The back propagation unit obtains the partial derivative of the mapping function to the inference result according to the inference result, the regularization term, and the network model, and adjusts the mapping function in combination with the learning rate to obtain a new mapping function;

[0077] like Figure 2 As shown, the image evaluation module includes:

[0078] An image quality evaluation unit, which obtains an image quality score of a generated image through an image evaluation algorithm;

[0079] The image decision unit skips, saves or updates the current image according to the inference result obtained by the image inference unit and the image quality score obtained by the image quality assessment module.

[0080] The slopes of each segment represented by the mapping function are randomly initialized within the brightness range of the original image.

[0081] The image mapping unit divides the brightness into segments according to a mapping function, and the brightness in each segment changes according to the same slope.

[0082] The image reasoning unit is a convolutional neural network, including but not limited to: LeNet, ResNet, DenseNet.

[0083] The regularization term is the inverse of the average of the absolute values ​​of the mapping function values.

[0084] The image quality assessment unit uses the reciprocal of LPIPS as a scoring standard to obtain an image quality score; the higher the image quality, the lower the LPIPS value, and the higher the image quality score.

[0085] The image decision unit compares the image quality score of the successful attack with the image quality score of the attack sample saved in the previous iteration, and selects the one with the higher score to save; if the current image attack is unsuccessful, it enters the next iteration.

[0086] For better understanding, the working process of the present invention will be described below.

[0087] like Figure 2 As shown, the image reading module first converts the input RGB image into a color space to obtain a representation of the image in the LAB space, and then separates the color and brightness to obtain an image brightness matrix and an image color matrix.

[0088] Preferably, the input image format is Tensor, and is converted from RGB space to LAB space image to obtain a Tensor in LAB color space.

[0089] The mapping division unit in the image mapping module obtains the effective value range of the image brightness according to the brightness matrix, and extracts the effective values ​​of the image brightness matrix to obtain the effective brightness matrix while saving the invalid part of the image.

[0090] In the image mapping unit, each parameter of the mapping function represents the slope when mapping the image to generate the attack sample. After the effective brightness matrix is ​​transformed by the mapping function, a new effective brightness matrix is ​​obtained, and then the effective brightness matrix is ​​integrated according to the invalid numerical information segmented by the mapping division unit to obtain a complete attack sample.

[0091] The image inference unit in the target network module performs inference classification based on the generated attack samples to obtain the classification result, that is, to judge whether the image is a specific object, such as a car, a person, a fish, etc. The regularization unit obtains the value of the regularization term based on the parameters of the mapping function. The back propagation unit derives the classification result and the regularization term to obtain the trend of modifying the mapping function parameters, and modifies the mapping function in combination with the learning rate.

[0092] Preferably, the regularization term is the inverse of the average value of the absolute values ​​of the mapping function parameters, so that there are as few parameters with values ​​of 0 as possible in the mapping function.

[0093] Next, the image mapping module is described in further detail.

[0094] The mapping division unit in the image mapping module obtains the effective range of the brightness matrix according to the maximum and minimum values ​​of the brightness matrix obtained by the image reading module in the brightness space.

[0095] Assume that the effective brightness range of the image is 0.25-0.75, such as Figure 3 shown.

[0096] According to the effective range, the mapping division unit cuts the brightness matrix to obtain an effective brightness matrix.

[0097] Specifically, after saving the minimum value of the entire brightness matrix, the minimum value is subtracted, and the portion of the new matrix with a value greater than 0 (and less than 0.5) is the effective brightness, and the new matrix is ​​the effective brightness matrix.

[0098] The image mapping unit in the image mapping module initializes the mapping function.

[0099] The mapping function is as follows:

[0100]

[0101]

[0102] In particular, the mapping function is a one-dimensional vector with K parameters, i.e., K floating point numbers, where K represents the number of segments into which the effective brightness is divided, and each parameter value represents the relative slope shared by each brightness segment after the division.

[0103] like Figure 3 As shown, for a picture with an effective brightness of 0.25-0.75, let K=4, then the parameters of the mapping function are represented as the portion from x=0.25 to x=0.75 in the image, where the mapping function parameters represent the relative slope of each segment.

[0104] Specifically, the first segment of the effective matrix is ​​mapped first during mapping, and 0 to 0.5 is divided into 4 segments, and the horizontal axis of each segment is 0.125.

[0105] Multiply the part of the effective brightness matrix greater than or equal to 0 and less than 0.125 by the first parameter to obtain the matrix M1 after mapping the corresponding position, that is, F θ (x k )=x k ·θ1,x k ∈[0,0.125); then subtract 0.125 from the effective brightness matrix, multiply the part of the new matrix after subtracting 0.125 that is greater than or equal to 0 and less than 0.125 by the second parameter, and add M1 calculated in the previous step to get M2, that is Until all four parts are mapped, a new effective brightness matrix M4 is obtained. After that, the obtained new effective brightness matrix M4 is scaled to be greater than or equal to 0 and less than or equal to 0.5, and a new effective brightness matrix is ​​obtained.

[0106] In particular, due to the characteristics of brightness, the mapping function maps the effective brightness of the image without being restricted by the slope, but the range of the mapped value is still within the valid range of the original image.

[0107] The image mapping unit maps the effective brightness matrix through a mapping function to obtain the effective brightness matrix after the attack, and supplements the invalid part of the image with the effective brightness matrix, that is, each parameter of the effective brightness matrix is ​​added with the minimum value of the original brightness to supplement it into the attack brightness matrix.

[0108] According to the image color matrix saved when the image reading module segments the original image, the attack brightness matrix is ​​combined with the image color matrix to obtain the original attack sample.

[0109] In particular, the mapping function is for the mapping of effective brightness, and Figure 3 The area outside the effective brightness (minimum value) is stored when the mapping division unit cuts the brightness matrix.

[0110] Taking any image as an example, further detailed description is given according to the following algorithm.

[0111] The algorithm of the image semantic adversarial attack sample generation method based on image brightness mapping in the present invention is as follows:

[0112]

[0113] When the program runs, it needs to input the original image x and the image label l.

[0114] Iteration number I, learning rate α, regularization rate η and mapping function F θ The three parameters of the number of segments K to be divided are also freely set by the user.

[0115] Image x first enters the image reading module. The image reading module reads images in png / jpg and other image formats through Python's OpenCV library, obtains a Numpy array represented by BGR, and then transforms the array dimension to RGB through the Numpy library, and the values ​​in the array represent the pixel values ​​of the original image. The image is then converted to Tensor format through the torch library, which significantly improves the efficiency of subsequent processing. Then, according to the formula for converting RGB color space to LAB color space, the Tensor represented in RGB color space is converted into a Tensor in LAB color space.

[0116] The input image x represented in LAB color space contains three dimensions, namely brightness With Color The image reading module converts the LAB color space The matrix is ​​divided into and in That is the brightness matrix of x.

[0117] The image division unit in the image mapping module is based on the brightness matrix The value range of the effective brightness matrix x is divided * .

[0118] The image mapping unit in the image mapping module initializes the mapping function according to the set k value

[0119] The mapping function is a one-dimensional vector with K parameters. K represents the number of segments into which the effective brightness is divided, and each parameter value represents the slope shared by each brightness segment after segmentation, and each parameter is a one-dimensional vector of random floating point numbers ranging from 0.8 to 1.2.

[0120] The image mapping unit in the image mapping module divides the effective brightness matrix into K equidistant segments, and each segment is multiplied by the mapping function The corresponding slope parameter is used to obtain the effective brightness matrix after the attack. Then, according to the range of the original effective brightness, the effective brightness matrix after the attack is scaled to ensure that it is the same as the original effective brightness range, and the new effective brightness matrix x is obtained. * 0.

[0121] The data saved when the effective brightness matrix is ​​divided according to the image division unit will be x * 0 restores to the full brightness matrix Then the image is segmented by the image reading module Matrix, which is the brightness matrix Add color part to generate attack image

[0122] The attack image x'0 is generated by the image mapping module and passed to the image reasoning unit in the target network module for processing.

[0123] The image inference unit is the target network of the attack, such as ResNet50 for image classification. After the image passes through ResNet50, a one-dimensional vector logit containing the number of classification parameters is obtained in the penultimate layer, which is the probability that the image belongs to each classification.

[0124] According to the input image label l, the probability of the corresponding classification in logit, that is, the probability of class l minus the maximum value except the probability of class l, is used to obtain the C&W loss L C&W (x' i-1 ,l).

[0125] In particular, since the mapping function F θ When the value in is 0, the image will be unnatural, so the regularization term sum(|θ|) / K is added to make F θ The smaller the absolute value of the parameter, the greater the overall loss, making the final F θThe probability of the value of 0 decreases; and the degree of regularization is adjusted by the hyperparameter regularization rate η, and in the present invention η=0.5.

[0126] In the target network, C&W The composite loss function combined with the regularization term is calculated with respect to the mapping function F θ The gradient of , we get the gradient g.

[0127] According to the gradient g and the learning rate α, The parameter θ0 is modified to obtain

[0128] pass Modify the effective brightness x of the original image * , and get the new effective brightness x * 1, and then restore to get a new attack sample x'1, and then repeat the above steps until the number of iterations reaches I, that is, the sample after the attack x'=x' i .

[0129] For each image that needs to be attacked, an attack is performed for the number of iterations I. After the iteration, if there is a sample with a successful attack, the image is successfully attacked, otherwise the attack fails.

[0130] like Figure 6 As shown in the figure, taking the bird image as an example, both PGD and OLF attack methods generate successful adversarial attack samples, that is, the target network is judged as not the bird. Compared with the original image, the attack image generated by the present invention is obviously the closest to the original image; and if the original image is not viewed, it can be observed that the image generated by the present method is the most natural; PGD is the classic L p Norm restriction attack, you can see that the background of the image it generates has obvious noise.

[0131] In particular, regarding the naturalness of an image, the most commonly used indicator is LPIPS. LPIPS is an evaluation indicator that mimics human visual perception and is used to measure the similarity of images in a manner consistent with human judgment. The lower the LPIPS value, the better the image quality. Therefore, the image quality assessment unit in the image assessment module of the present invention uses LPIPS as an evaluation indicator, and the results obtained on commonly used data sets are better than existing methods.

[0132] Specifically, Figure 6 The LPIPS of the images shown are: 0.1480 (PGD) and 0.0224 (OLF).

[0133] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A system for generating image semantic adversarial attack samples based on image brightness mapping, characterized in that: include: The image reading module is used to read the original image information, convert the input RGB image into a LAB space image, and separate the color and brightness of the image to obtain the image brightness matrix and the image color matrix; An image mapping module is used to intercept the effective brightness matrix and the invalid part of the image according to the brightness matrix of the original image, and use a mapping function to map the effective brightness matrix. After the mapping is completed, it is integrated with the invalid part of the image to obtain the original attack sample; The image map module includes: A mapping division unit, used for dividing the effective brightness matrix based on the effective brightness value of the image and the invalid part of the image according to the brightness range of the original image; An image mapping unit is used to initialize a mapping function, map the effective brightness matrix according to the mapping function, integrate the mapped effective brightness matrix and the invalid part of the image obtained in the mapping division unit, and obtain an original attack sample; the parameter value of the mapping function is set to a random floating point number of 0.8 to 1.2; the image mapping unit divides the brightness into segments evenly according to the mapping function, and the brightness in each segment changes the value at the same slope according to the corresponding parameter; The target network module is used to put the original attack sample into the target network for inference, and after obtaining the inference result, back-propagate, adjust the mapping function, and obtain a new mapping function; the target network module includes: An image reasoning unit, which puts the original attack sample into the image reasoning unit for reasoning to obtain a reasoning result; the image reasoning unit is a convolutional neural network, including LeNet, ResNet, and DenseNet; the regularization term in the target network module is the opposite of the average value of the absolute value of each value of the mapping function; Regularization unit, used to make the loss value contain a term whose slope of the mapping function is not zero; The loss function after adding the regularization term is: Loss=L C&W (x,l)-ηsum(|θ|) / K Among them, L C&W (·) is the C&W loss, which is the value of the class corresponding to label l in the inference result obtained by the image inference unit for the input image x minus the maximum value outside the class, sum(|θ|) / K is the regularization term; K represents the number of segments for segmenting effective brightness; η is the regularization rate, which is used to adjust the degree of regularization, that is, the proportion of the regularization term in the loss function, η=0.5; The back propagation unit obtains the partial derivative of the mapping function to the inference result according to the obtained inference result and the value of the regularization term obtained by the regularization unit, and adjusts the mapping function in combination with the learning rate to obtain a new mapping function; the slopes of each segment represented by the mapping function are randomly initialized within the brightness range of the original image; the image mapping unit divides the brightness into segments according to the mapping function, and the brightness in each segment changes according to the same slope; the learning rate is 0.3; According to the partial derivative, that is, the degree of influence of the parameters of the mapping function on the final result, the direction of parameter modification is determined, that is, whether to increase or decrease the parameter value; the partial derivative is obtained by calculation; after the direction of parameter adjustment is determined by the partial derivative, the size of each step of adjustment is the learning rate; if the parameter θ is obtained after a certain calculation k The partial derivative is positive, and the learning rate is 0.

3. Then the parameter θ after this iteration k The value of is adjusted by adding 0.3; the parameter θ k Indicates the parameters corresponding to the kth segment; An image evaluation module is used to select to skip, save or update the current original attack sample according to the image reasoning result and the image quality score; the image evaluation module includes: An image quality evaluation unit obtains an image quality score of a generated image through an image evaluation algorithm; the image quality evaluation unit uses the reciprocal of LPIPS as a scoring standard to obtain an image quality score; the higher the image quality, the lower the LPIPS value, and the higher the image quality score; the image decision unit compares the image quality score of the successfully attacked image obtained through the image quality evaluation unit with the image quality score of the attack sample saved in the previous iteration, and selects the image with the higher image quality score to save; if the current image attack is unsuccessful, it is saved as a failed image; in the image classification task, the successful attack means that the category and label of the image after the attack are different from the image inferred by the image reasoning module; the unsuccessful attack means that the category and label of the image after the attack are the same as the image inferred by the image reasoning module; The image decision unit skips, saves or updates the current original attack sample image according to the inference result obtained by the image inference unit and the image quality score obtained by the image quality assessment module.

2. A method for generating an attack sample using the generation system as claimed in claim 1, characterized in that: The generation method comprises the following steps: Step 1: Input the original image into the image reading module, convert the original image into Tensor format, and convert it from RGB color space to LAB color space to obtain the brightness matrix and color matrix of the input image; Step 2: The image mapping module extracts the effective brightness matrix from the original image brightness matrix obtained in step 1, and maps the effective brightness matrix using a mapping function. After the mapping is completed, it is integrated with the invalid part of the image to obtain the original attack sample; Step 3: Input the original attack sample into the target network module, use the image inference unit to perform inference and obtain the inference result, and the back propagation unit corrects the mapping function according to the classification result obtained by inference and the value of the regularization term obtained by the regularization unit and the learning rate; Step 4: The image evaluation module selects to skip, save or update the original attack sample obtained in step 2 by comparing the inference result in step 3 and the image quality score obtained by the image quality evaluation module; Step 5: Repeat steps 2 to 4 to obtain the final attack sample.

3. The generation method according to claim 2, characterized in that: In step 1, the image reading module reads the image through the OpenCV library of python, adjusts the image format from BGR to RGB using the Numpy library, and then obtains the brightness information and color information of the image through the conversion formula between RGB space and LAB space; the original image is converted into Tensor format through the torch library of python; the formula for converting the data in Tensor format from RGB color space to LAB color space is as follows: Among them, R, G, B represent the three channel components of the original image in the RGB color space, and they are all two-dimensional matrices with the same size as the image; X, Y, Z are the three channel components represented in the XYZ color space obtained by representing the image in the RGB color space; X n ,Y n ,Z n is a fixed value, X n =0.95047,Y n =1.0,Z n =1.08883; By performing matrix operations on the XYZ color space, the representation of the original image in the LAB color space is obtained.

4. The generation method according to claim 2, characterized in that: In step 2, the setting of the effective brightness is determined according to the maximum and minimum values ​​of the input image in the brightness space; that is, in the brightness space, the part of the matrix value greater than 0 after subtracting the minimum value in the brightness matrix is ​​the effective brightness of the input image; after the effective brightness matrix is ​​intercepted, the invalid part of the image is saved at the same time; The mapping function has K parameters, i.e., a one-dimensional vector of K floating point numbers, as shown below: Among them, K represents the number of segments for dividing the effective brightness, θ k represents the parameter corresponding to the kth segment, x k Represents the brightness value of any position in the original image, F θ (x k ) is the new brightness value of the corresponding position of the image after mapping.

5. The generation method according to claim 2, characterized in that: In step 3, the image inference unit is a convolutional neural network, which accepts input (b, 3, W, H), extracts features through several convolutional layers and several pooling layers, and then connects to a fully connected layer with a classification number c for classification, where b is the batch number, 3 represents RGB three dimensions, W is the image width, and H is the image height; in the image classification task, after one or b pictures that have been read into the Tensor format are input into the target network, a 1-dimensional vector representing the category to which the picture belongs judged by the target network is obtained; the number of categories of the 1-dimensional vector is c parameters, which respectively represent the possibility of the category to which the picture belongs inferred by the target network, and the category with the highest value is the category of the picture inferred by the target network; The network characteristics of the convolutional neural network are as follows: for an input image, after extracting features through one or more convolutional layers and pooling layers, one or more fully connected layers are connected, and the number of parameters in the output layer is the number of target categories.

6. The generation method according to claim 2, characterized in that: In step 4, for the attack image generated in a certain iteration, the generated image is put into the image reasoning module in step 3 to check whether it can cause the image reasoning module to misclassify it. If it is misclassified, it indicates that the attack is successful. For the image with a successful attack, its score in the image evaluation module is compared with the corresponding score of the currently saved image, that is, the attack sample saved in the previous iteration. If the score is higher, the new image is saved, otherwise the new image is discarded and the next iteration is entered.

Images