[0039]The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0040] In the description of the present invention, it should be noted that "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. The indicated direction or positional relationship is based on the direction or positional relationship described in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the indicated device or element must have a specific orientation or a specific orientation. construction and operation, and therefore should not be construed as limiting the invention. Furthermore, ordinal numbers (eg, "first and second," "first through fourth," etc.) are used to distinguish between objects, are not limited to that order, and should not be construed to indicate or imply relative importance.
[0041] In the description of the present invention, it should be noted that, unless otherwise expressly specified and limited, “installation”, “connection” and “connection” should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection Connection, or integral connection; it can be mechanical connection or electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be internal communication between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood in specific situations.
[0042] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0043] In one example, as figure 1 As shown, a method for generating adversarial samples with multi-pass aggregation specifically includes the following steps:
[0044] S1: establish multiple model paths, each model path includes multiple neural network models connected in sequence, the same level nodes in each model path use the same neural network model, and the neural network models corresponding to the same level nodes are adjacent and interconnected;
[0045] S2: Add random disturbance information to the original image to obtain multiple first disturbance images;
[0046] S3: Input the original image into the first model path, and simultaneously input multiple first disturbed images into other model paths, calculate the gradient of each neural network model, and account for all neural networks of the same level node according to the predicted similarity of the current neural network model. The proportion of the total similarity of the model, the gradient of each neural network model is subjected to adaptive weight aggregation processing, and the image samples generated by each neural network model are updated according to the gradient obtained by the adaptive weight aggregation processing, and this step is repeated multiple times to make the first The model paths output the final adversarial examples.
[0047] As an option, step S2 may be performed prior to step S1, or steps S1 and S2 may be performed simultaneously.
[0048] Further, in step S1, the neural network model is specifically a neural network model capable of image processing, and the neural network model is randomly selected from the model pool during the model path establishment process. In this example, the model pool is a classifier of the ImageNet dataset, including Not limited to neural network models such as Inceptionv3, Inceptionv4, InceptionResNetv2, Xception, ResNetv2-101. Each neural network model corresponds to a node of the current model path, and each node is an iteration of the neural network model; peer nodes are the neural network models in the same position in each model path, for example, the first model path includes sequential connections. Neural network model a and neural network model b, the second model path includes sequentially connected neural network model c and neural network model d, and the third model path includes sequentially connected neural network model e and neural network model f, At this time, the neural network model a, the neural network model c, and the neural network model e are in the same level nodes, and the neural network model b, the neural network model d, and the neural network model f are in the same level nodes. Further, the neural network models corresponding to the nodes at the same level are adjacent and interconnected, that is, the neural network models are interconnected with the neural network models of the same level in the adjacent model paths, such as the neural network model a, the neural network model c, and the neural network model e are interconnected in turn, The neural network model b, the neural network model d, and the neural network model f are interconnected in turn. Preferably, the types of the multiple neural network models in each model pathway are different. Of course, as an option, the types of the multiple neural network models in each model pathway can be the same. In this example, the more model paths, and the more nodes each path (model path) includes in the neural network model, the stronger the generalization of the final adversarial samples generated.
[0049] Further, in step S2, for each type of disturbance information added to the original image, a corresponding first disturbance image is obtained. Preferably, the random disturbance information added to the original image is different, so as to obtain more samples of the original image, synthesizing more external disturbance factors to the greatest extent, and improving the generalization of image samples.
[0050] Further, in step S3, the predicted similarity is calculated according to the difference between the predicted value of the current neural network model and the real label value; in the adaptive weight aggregation process, the weight is specifically that the similarity of the node neural network model accounts for the same level of nodes. The proportion of the total similarity of all neural network models, through the adaptive weight aggregation processing step, the disturbance information of the generated image samples is updated, so that the disturbance factors of different types and sizes can obtain the weights based on their own disturbances, thereby enabling the adversarial samples It can further improve the generalization of adversarial samples by synthesizing various disturbance factors without destroying the image quality of the naked eye.
[0051] Further, in step S3, the final adversarial sample with the strongest generalization performance is output through the first model path (the model path that receives the original image), because the disturbance information update of each level node in each model path is the same as the previous one. On the basis of level 1 (level 1 nodes are updated on the input image), the first perturbed image input by the neural network model in other paths may have a gap with the original image in terms of visual perception, resulting in the generated adversarial samples reaching If the attack effect is not good, such as cropping in geometric perturbation, the first perturbed image obtained after cropping the original image may still be recognized by the classifier, but the cropping traces can be visually observed by the naked eye, which leads to the input of the first perturbed image. The adversarial sample generated by a perturbed image path has obvious clipping traces, which cannot meet the condition that the adversarial sample should be similar to the original image in terms of visual perception.
[0052] The invention uses various types of disturbances to process images, so that the generated confrontation samples have stronger generalization, and the attack power of the confrontation samples on different models is improved. The anti-interference ability of a variety of different disturbance factors results in a model with high defense power. Compared with adding the gradient aggregation disturbance of different models to the original image, the adversarial sample of this application is more sensitive to external disturbance factors. At the same time, the original image and its multi-samples (the first perturbed image) are input into multiple model paths to train the generation model of the adversarial samples, which effectively reduces the over-fitting caused by the simplification of samples while taking into account the disturbances of different sizes and types. . Further, the present application performs adaptive weight aggregation on the gradient information of multiple paths, and acts on the image update of each node of each path, and various disturbance factors are fitted, so that the generalization of the generated adversarial samples can be improved. Because the adaptive weight is used to aggregate the gradient, the defense ability of the model trained by the adversarial sample generation against disturbances of different types and sizes is further improved. Finally, a multi-node neural network model is used to perform gradient iteration on the image on multiple model paths, which further improves the generalization of the final adversarial samples generated.
[0053] It should be noted that only a small proportion of disturbance information is added to the original image in this application, and the first disturbance image is input into the neural network model, and the recognition accuracy of the model for the input sample (the first disturbance image) is still not high. will be greatly reduced, so the first disturbed image is generally not used as an adversarial sample to attack the model. Therefore, this application further fits the various disturbance factors of the original image and the first disturbed image through multiple model paths to generate high attacks. The final adversarial example of the force.
[0054] In an example, adding random perturbation information to the original image includes:
[0055] Perform geometric transformation processing and/or color processing and/or image fusion processing and/or filtering processing on the original image to obtain a first disturbed image. Among them, geometric transformations include clipping, zooming in, zooming out, translation, staggered transformation, rotation transformation, etc.; color processing is to modify the RGBA value of the image, where R represents red, G represents green, B represents blue, and A represents transparency Image fusion is the fusion of images through algorithms, such as weighted average method, image fusion method based on wavelet transform, etc.; image filtering includes mean value filtering, Gaussian filtering, sharpening methods, etc.
[0056] In an example, the calculation formula of the adaptive weight aggregation process is:
[0057]
[0058] in, p Indicates the model channel number; n Indicates the upper limit of the model path; t Indicates the node number in the model path; express t Gradient aggregation of all model paths of level nodes; means the first p the first part of the model pathway t similarity of level nodes; for t The sum of the similarity of all model paths of the level node; represents the gradient symbol; means the first p the first part of the model pathway t The image input by the level node; CE represents the cross loss entropy; y ture Image representing the input neural network model x the corresponding true label value; M represents the neural network model; Represents a neural network model M Predicted value for the input image.
[0059] In an example, the parameters of the model algorithm need to be defined. In this application, the perturbation size is 16, that is, the maximum difference ε of the infinite norm between the generated adversarial sample and the original image is 16, and the number of iterations is T ,Right now T level node, as an option, T =10, the learning rate at this time. On this basis, the loss of the model to the input image is further calculated. The specific calculation formula is:
[0060]
[0061] in, is the first in the real label a code; is the first in the predicted value a code; C Represents the number of categories in the label.
[0062] In an example, the calculation formula of the ratio of the predicted similarity of the current neural network model to the total similarity of all neural network models of the same level node is:
[0063]
[0064] where ||*|| represents the norm, ||*|| 1 Represents the sum of the absolute values of the vector elements.
[0065] In an example, the calculation formula for updating the image samples generated by each neural network model according to the gradient obtained by the adaptive weight aggregation process is:
[0066]
[0067] in, means the first p the first part of the model pathway t+1 The image input by the level node; σ represents the disturbance size; express t Gradient aggregation of all model paths at level nodes.
[0068] In an example, after updating the image samples generated by each neural network model according to the gradient obtained by the adaptive weight aggregation process, the method further includes:
[0069] Constrain the image samples to obtain temporary adversarial samples. The specific calculation formula is:
[0070]
[0071] in, clip Indicates the constraint range.
[0072] The above examples are combined to obtain the preferred examples of this application, such as figure 2 As shown, a multi-pass aggregation-based adversarial sample generation method includes the following steps:
[0073] S1’: Randomly select the original image to add random disturbance information, and each time a disturbance information is added, a first disturbance image is obtained, the original image and the disturbed image have a total of n open;
[0074] S2': Build n Model paths: Each model path consists of 10-level nodes, the neural network model of each node is randomly selected from the model pool, and the same level nodes of all model paths use the same neural network model;
[0075] S3': will n input images separately n A new level node of the path, and calculate the gradient;
[0076] S4': will n The gradients of the nodes at the same level of each path are processed by adaptive weight aggregation;
[0077] S5': update according to the aggregated gradient n image;
[0078] S6': Execute S3'-S5', cycle 10 times, reach the 10-level node, and output the final adversarial sample of the original image in the first model path.
[0079] The present invention also includes a model training method based on multi-channel aggregation of adversarial samples, which has the same inventive concept as the above-mentioned method for generating high-transfer adversarial samples, and specifically includes:
[0080] The neural network model is trained according to the adversarial samples, so that the model can learn the features of the adversarial samples and accurately classify the adversarial samples, and obtain a model with high defense power. Among them, the model can learn the perturbation features of the adversarial samples that are different from the original images, and then correct the classification results to achieve accurate classification and improve the security performance of the neural network model. The invention generates a final confrontation sample with high generalization to attack the black box model, and the generated confrontation sample also has a high attack success rate for the black box model.
[0081] The present invention also includes a multi-pass aggregation adversarial sample generation system, the system includes a model unit and an iterative calculation unit.
[0082] like image 3 shown, the model unit includes n Each model path includes 10 neural network models connected in sequence, the same level nodes in each model path use the same neural network model, and the neural network models corresponding to the same level nodes are adjacent and interconnected; the first model path receives the original image, and other model paths respectively receive the first perturbed image, and finally, by aggregating all model paths, the first model path is assisted to generate the final adversarial sample; wherein random perturbation information is added to the original image to obtain the first perturbed image;
[0083] The iterative calculation unit is used to calculate the gradient of each neural network model. According to the proportion of the predicted similarity of the current neural network model to the total similarity of all neural network models of the same node, the gradient of each neural network model is subjected to adaptive weight aggregation processing. , and update the image samples generated by each neural network model according to the gradient obtained by the adaptive weight aggregation processing, and iteratively calculate for many times, and then generate the final confrontation sample.
[0084] In an example, the system further includes a model pooling unit for storing neural network models of the ImageNet dataset, including but not limited to neural network models such as Inceptionv3, Inceptionv4, InceptionResNetv2, Xception, and ResNetv2-101.
[0085] The present application also includes a storage medium, which has the same inventive concept as a method for generating an adversarial sample with multi-pass aggregation composed of any one or more of the above examples, and has computer instructions stored thereon, and the computer instructions are executed when running. The steps of the above-mentioned multi-pass aggregation adversarial sample generation method.
[0086] Based on this understanding, the technical solution of this embodiment can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution, and the computer software product is stored in a storage medium, Several instructions are included to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, removable hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.
[0087] The present application also includes a terminal, which has the same inventive concept as a method for generating an adversarial sample with multi-pass aggregation composed of any one or more of the above examples, and includes a memory and a processor, and the memory stores data that can be stored in the The computer instructions run on the processor, and when the processor runs the computer instructions, the steps of the above-mentioned method for generating an adversarial sample for multi-pass aggregation are performed. The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.
[0088] Each functional unit in the embodiments provided by the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
[0089] The above specific embodiments are detailed descriptions of the present invention, and it cannot be assumed that the specific embodiments of the present invention are limited to these descriptions. Some simple deductions and substitutions should be considered as belonging to the protection scope of the present invention.
