Method, device, apparatus, and readable storage medium for generating adversarial samples

Abstract

The present invention discloses an adversarial sample generation method, apparatus, device, and readable storage medium. The method comprises: obtaining an original image, performing diversity construction processing on the original image, and obtaining a diverse image to be processed; obtaining N alternative attack network models associated with a target network model to be attacked, and generating adversarial perturbations for the image to be processed using the N alternative attack network models; N is a positive integer; the alternative attack network models and the target network model have the same task objective; adding adversarial perturbations to the image to be processed based on perturbation weights for the foreground region and background region of the original image, and obtaining an adversarial sample image; and using the adversarial sample image to conduct an adversarial attack on the target network model. The present invention can improve the authenticity of adversarial samples and the effectiveness of model security assessments.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, device, and readable storage medium for generating adversarial samples. Background Art

[0002] With the development of deep learning technology, AI (Artificial Intelligence) products have gradually been put into practical use and are playing a vital role in all areas of daily life. Against this backdrop, the security of AI products is particularly important. However, the rise of adversarial attack techniques allows criminals to influence the output of AI models by simply adding perturbations to image samples that are imperceptible to the naked eye. This poses a significant security threat to a variety of fields, including autonomous driving, smart healthcare, and smart retail. Therefore, how to effectively assess the security of AI products before they are put into use has become an urgent task.

[0003] Currently, the industry primarily uses white-box and black-box adversarial attacks to generate adversarial examples. The security of AI models is then assessed based on the success rate of these attacks. White-box adversarial attacks involve directly generating adversarial examples through methods like gradient backpropagation, while obtaining the AI model's structure, parameters, and gradients. Black-box adversarial attacks involve generating adversarial examples based on the changes in the model's output for different inputs by performing black-box queries on the model without knowing its internal information. However, in reality, attackers often lack access to any internal information, such as the model's structure, parameters, and gradients. Furthermore, most AI products are equipped with comprehensive risk control policies, such as limiting the number of daily submissions per account, making it impossible for attackers to conduct large numbers of queries. Consequently, both adversarial attack methods struggle to realistically simulate the potential threats that AI models face in the real world, limiting the effectiveness of their assessments. Summary of the Invention

[0004] The embodiments of the present application provide a method, apparatus, device, and readable storage medium for generating adversarial samples, which can improve the authenticity of adversarial samples and the effectiveness of model security assessment.

[0005] On the one hand, an embodiment of the present application provides a method for generating adversarial samples, including:

[0006] Acquire an original image, perform diversity construction processing on the original image, and obtain a to-be-processed image with diversity;

[0007] Obtain N alternative attack network models associated with the target network model to be attacked, and generate adversarial perturbations for the image to be processed through the N alternative attack network models; N is a positive integer; the alternative attack network models and the target network model have the same task objectives;

[0008] According to the perturbation weights of the foreground area and the background area of the original image, the adversarial perturbation is added to the image to be processed to obtain an adversarial sample image; the adversarial sample image is used to conduct an adversarial attack on the target network model.

[0009] An embodiment of the present application provides, on one hand, a device for generating adversarial samples, including:

[0010] An image construction module is used to obtain an original image and perform a diversity construction process on the original image to obtain a diverse image to be processed;

[0011] A perturbation generation module is used to obtain N alternative attack network models associated with the target network model to be attacked, and generate adversarial perturbations for the image to be processed through the N alternative attack network models; N is a positive integer; the alternative attack network models and the target network model have the same task objectives;

[0012] The perturbation adding module is used to add adversarial perturbations to the image to be processed based on the perturbation weights of the foreground area and the background area of the original image to obtain adversarial sample images; the adversarial sample images are used to conduct adversarial attacks on the target network model.

[0013] The image construction module includes:

[0014] An image acquisition unit, configured to acquire an original image;

[0015] A scaling unit is used to scale the original image to obtain a scaled image; the size of the scaled image is smaller than the size of the original image;

[0016] The interpolation filling unit is used to perform interpolation filling processing on the scaled image to obtain an image to be processed; the size of the image to be processed is the same as the size of the original image.

[0017] Among them, the disturbance generation module includes:

[0018] The first iterative processing unit is configured to generate, in an i-th training iteration, a training perturbation corresponding to the training image in the i-th training iteration through N replacement attack network models; i is a positive integer; and the training image in the first training iteration is the image to be processed;

[0019] a first image updating unit, configured to update the training image in the i-th training iteration according to the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration, to obtain a training image in the i+1-th training iteration, if i is less than the iteration number threshold;

[0020] A first momentum updating unit is configured to update the accumulated momentum in the i-th training iteration according to the training perturbations corresponding to the training images in the i-th training iteration to obtain the accumulated momentum in the i+1-th training iteration;

[0021] A second iterative processing unit is configured to generate, in the (i+1)th training iteration, a training perturbation corresponding to the training image in the (i+1)th training iteration through N replacement attack network models;

[0022] The first disturbance determination unit is configured to determine the training disturbance corresponding to the training image in the i+1th training iteration as the adversarial disturbance if i+1 is equal to the iteration number threshold.

[0023] The first iterative processing unit includes:

[0024] The model output subunit is used to input the training images in the i-th training iteration into N replacement attack network models respectively, and obtain the result vectors output by the N replacement attack network models respectively;

[0025] The vector averaging subunit is used to average the result vectors output by the N alternative attack network models to obtain an average result vector;

[0026] A loss determination subunit, configured to determine the loss function associated with the training image in the i-th training iteration based on the average result vector and the true label;

[0027] The training perturbation generating subunit is used to determine the gradient of the loss function with respect to the training image in the i-th training iteration, and to generate the training perturbation corresponding to the training image in the i-th training iteration along the reverse direction of the gradient.

[0028] Wherein, the first momentum updating unit includes:

[0029] A remainder determination subunit is used to obtain a momentum reset parameter, and divide the default number of iterations by the momentum reset parameter to obtain a target remainder;

[0030] A momentum update subunit, configured to add the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration to obtain the accumulated momentum in the i+1-th training iteration if the target remainder does not satisfy the momentum reset condition;

[0031] The momentum update subunit is further configured to reset the accumulated momentum in the i-th training iteration if the target remainder satisfies the momentum reset condition, thereby obtaining the accumulated momentum in the i+1-th training iteration.

[0032] Among them, the disturbance generation module includes:

[0033] The third iterative processing unit is configured to generate, in the i-th training iteration, a training perturbation corresponding to the training image in the i-th training iteration through N replacement attack network models; i is a positive integer; the training image in the first training iteration is the image to be processed;

[0034] a first loss determination unit, configured to determine a loss function value for an i-th training iteration based on the N alternative attack network models and the training image in the i-th training iteration;

[0035] a second image updating unit, configured to update the training image in the i-th training iteration according to the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration, to obtain a training image in the (i+1)-th training iteration, if the loss function value of the i-th training iteration is less than the loss threshold;

[0036] A second momentum updating unit is configured to update the accumulated momentum in the i-th training iteration according to the training perturbations corresponding to the training images in the i-th training iteration to obtain the accumulated momentum in the i+1-th training iteration;

[0037] a fourth iterative processing unit, configured to generate, in the (i+1)th training iteration, a training perturbation corresponding to the training image in the (i+1)th training iteration through N replacement attack network models;

[0038] a second loss determination unit, configured to determine a loss function value for an i+1th training iteration based on the N alternative attack network models and the training image in the i+1th training iteration;

[0039] The second disturbance determination unit is configured to determine the training disturbance corresponding to the training image in the (i+1)th training iteration as the adversarial disturbance if the loss function value of the (i+1)th training iteration is greater than or equal to the loss threshold.

[0040] The adversarial sample generation device further includes:

[0041] The segmentation module is used to perform saliency segmentation processing on the original image to obtain the foreground area position information and the background area position information;

[0042] Perturbation adding module, including:

[0043] an area determination unit, configured to determine a foreground area to be processed of the image to be processed according to the foreground area position information, and to determine a background area to be processed of the image to be processed according to the background area position information;

[0044] a foreground adding unit, configured to update the foreground region to be processed of the image to be processed according to the perturbation weight of the foreground region and the adversarial perturbation, to obtain a foreground adversarial image;

[0045] a background adding unit, configured to update the background region to be processed of the image to be processed according to the disturbance weight of the background region and the adversarial disturbance, to obtain a background adversarial image;

[0046] The fusion unit is used to fuse the foreground adversarial image and the background adversarial image to obtain an adversarial sample image.

[0047] The adversarial sample generation device further includes:

[0048] The attack module is used to conduct adversarial attacks on the target network model through adversarial sample images and obtain the attack success rate;

[0049] The security assessment module is used to evaluate the security level of the target network model based on the attack success rate.

[0050] On one hand, an embodiment of the present application provides a computer device, including: a processor, a memory, and a network interface;

[0051] The above-mentioned processor is connected to the above-mentioned memory and the above-mentioned network interface, wherein the above-mentioned network interface is used to provide a data communication network element, the above-mentioned memory is used to store a computer program, and the above-mentioned processor is used to call the above-mentioned computer program to execute the method in the embodiment of the present application.

[0052] On one hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the method in the embodiment of the present application.

[0053] On the one hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in the embodiment of the present application.

[0054] The embodiment of the present application can obtain the original image, perform diversity construction processing on the original image, obtain a diverse image to be processed, and then obtain N alternative attack network models associated with the target network model to be attacked, and generate adversarial perturbations for the image to be processed through these N alternative attack network models; finally, according to the perturbation weights of the foreground area and the perturbation weights of the background area of the original image, the adversarial perturbations are added to the image to be processed to obtain an adversarial sample image. Wherein, N is a positive integer; the alternative attack network model and the target network model have the same task objectives; the adversarial sample image is used to conduct an adversarial attack on the target network model. Using the method provided by the present application, adversarial sample images are generated through N alternative attack network models. Without relying on the target network model for any query, the target network model can be subjected to an adversarial attack through the adversarial sample image, which can improve the authenticity of the adversarial sample, more effectively simulate the attack threats that the target network model may suffer, and further improve the effectiveness of the security assessment of the target network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0056] Figure 1 This is a schematic diagram of a network architecture provided by an embodiment of the present application;

[0057] Figure 2a This is a schematic diagram of a scenario for generating adversarial samples provided by an embodiment of the present application;

[0058] Figure 2b This is a schematic diagram of a scenario for generating adversarial samples provided by an embodiment of the present application;

[0059] Figure 2c This is a schematic diagram of a scenario for generating adversarial samples provided by an embodiment of the present application;

[0060] Figure 3 This is a flowchart of a method for generating adversarial samples provided in an embodiment of the present application;

[0061] Figure 4 1 is a flow chart of a method for generating an anti-disturbance signal provided in an embodiment of the present application;

[0062] Figure 5 This is a schematic diagram of a disturbance-adding scenario provided by an embodiment of the present application;

[0063] Figure 6This is a schematic diagram of the overall framework of an adversarial sample generation method provided in an embodiment of the present application;

[0064] Figure 7 Schematic diagram of the structure of an adversarial sample generation device provided in an embodiment of the present application;

[0065] Figure 8 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0067] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive field within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0068] The solution provided in the embodiment of the present application belongs to the field of artificial intelligence, namely computer vision technology (CV) and machine learning (ML).

[0069] Computer vision (CV) is the science of making machines "see." Specifically, it refers to machine vision, where cameras and computers replace the human eye in identifying and measuring objects, performing further image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0070] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0071] See Figure 1 , is a schematic diagram of a network architecture provided by an embodiment of the present application. Figure 1 As shown, the network architecture may include a business server 1000 and a background server cluster, wherein the background server cluster may include multiple background servers, such as Figure 1 As shown, it may specifically include a backend server 100a, a backend server 100b, a backend server 100c, ..., a backend server 100n. Figure 1 As shown, the backend server 100a, the backend server 100b, the backend server 100c, ..., the backend server 100n can respectively establish a network connection with the above-mentioned business server 1000, so that each backend server can exchange data with the business server 1000 through the network connection.

[0072] like Figure 1As shown, each backend server in the embodiment of the present application corresponds to a terminal device, and can respond to the adversarial sample generation instruction for the target network model initiated by the terminal device, initiate a business request for adversarial sample generation for the target network model to the business server 1000, and send original business data to the business server 1000. The target network model can be a neural network model for image processing. The original business data can include the original image, the task goal of the target network model, and the real label related to the original image and the task goal. The task goal of the target network model can be to classify the input image, detect the target, perform semantic segmentation or instance segmentation, etc. After the business server 1000 receives the business request from a backend server, it will perform adversarial sample generation processing based on the original business data to obtain an adversarial sample image. The adversarial sample image is used to conduct an adversarial attack on the target network model, and then the security level of the target network model is evaluated according to the attack success rate of the adversarial attack.

[0073] It is understandable that the method provided in the embodiments of the present application can be executed by a computer device, and the computer device can be the business server 1000 described above. The business server 1000 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0074] Among them, the background server and the business server can be directly or indirectly connected through wired or wireless communication, and this application does not impose any restrictions on this.

[0075] in, Figure 1 The terminal devices corresponding to the backend server cluster in the system may include mobile phones, tablet computers, laptops, PDAs, smart speakers, mobile Internet devices (MID), POS (Point of Sales) machines, wearable devices (such as smart watches, smart bracelets, etc.), vehicle-mounted terminals, etc.

[0076] For easier understanding, see Figure 2a-2c , Figure 2a-2c This is a schematic diagram of a scenario for generating adversarial samples provided by an embodiment of the present application. Figure 2aAs shown, tester A needs to perform a security assessment on the target network model, assuming that the task goal of the target network model is classification, and the target network model is a classification model for identifying animals in the input image. At this time, tester A needs to generate adversarial sample images for the target network model. Tester A can send a datagram to the backend server 21 (which can be the above-mentioned 21) through the terminal device 20. Figure 1 Any backend server in the backend server cluster shown, for example, backend server 100a) sends an adversarial sample generation instruction. After receiving the adversarial sample generation instruction, the backend server 21 can obtain the original image 200 that matches the target network model and the real label of the original image 200 related to the task target according to the adversarial sample generation instruction. Figure 2a As shown, the original image 200 that matches the target network model should be an animal image. For example, the original image 200 can be an image of a puppy. It can be understood that the true label of the original image 200 related to the target network model is a puppy. That is, the original image 200 is input into the target network model, and the target network model can recognize that the animal in the original image 200 is a puppy. Then, the background server 21 generates an instruction based on the adversarial sample and sends it to the business server 22 (which can be the above-mentioned Figure 1 The business server 1000 shown in FIG. 1 initiates a business request for generating adversarial samples for the target network model, and then sends the acquired original image 200 and the true label corresponding to the original image 200 as raw business data to the business server 22. It should be noted that the backend server 21 can acquire multiple original images in batches, and then send the multiple original images and the true label corresponding to each original image as raw business data to the business server 22.

[0077] After receiving the service request, the service server 22 will generate adversarial samples based on the original service data to obtain adversarial sample images. Figure 2bAs shown, after the service server 22 obtains the original image 200, it performs a diversity construction process on the original image 200 to obtain a diverse image to be processed 201. The diversity construction process is performed by performing special processing on the given original image, converting it into a variety of images to be processed that are different from the original image while maintaining its original main features. After obtaining the image to be processed 201, the service server 22 generates an adversarial sample image 202 based on the image to be processed 201. This can improve the transferability of the generated adversarial sample image 202 and thereby enhance the generalization of the adversarial attack. The service server 22 adds an adversarial perturbation to the image to be processed 201 to obtain the adversarial sample image 202. The adversarial perturbation is a tiny noise that is imperceptible to the human eye. While the image after adding the adversarial perturbation does not affect human recognition, it can easily cause the network model to misjudge. The service server 22 can generate the adversarial perturbation using methods such as gradient, optimization, or decision surface. In the embodiment of the present application, the adversarial perturbation is generated by using the image to be processed 201 and a replacement attack network model, without any input or output to the target network model. The task objectives of the substitute attack network model and the target network model are the same. In addition, the service server 22 can obtain N substitute attack network models and the image to be processed 201 to generate adversarial disturbances through an integrated learning strategy. Figure 2bAs shown, assume that the service server 22 obtains alternative attack network model 1, alternative attack network model 2, and alternative attack network model 3. It is understood that alternative attack network model 1, alternative attack network model 2, and alternative attack network model 3 are all classification models that can identify animals in input images, but they differ in structure, parameters, etc. In short, when the image 201 to be processed is input into alternative attack network model 1, alternative attack network model 2, and alternative attack network model 3, respectively, alternative attack network model 1, alternative attack network model 2, and alternative attack network model 3 can all identify that the animal in the image 201 to be processed is a puppy. The service server 22 can generate adversarial perturbations based on the image 201 to be processed and the alternative attack network models 1, alternative attack network model 2, and alternative attack network model 3. It is understood that the more adversarial perturbations added to the image 201 to be processed, the better the adversarial attack effect on the resulting adversarial sample image 202. Therefore, after obtaining the adversarial perturbation, the service server 22 can obtain the perturbation weight of the foreground area and the perturbation weight of the background area, wherein the perturbation weight of the foreground area is greater than the perturbation weight of the background area. Then, the service server 22 can add the adversarial perturbation to the image to be processed 201 based on the perturbation weight of the foreground area and the perturbation weight of the background area to obtain the adversarial sample image 202. The perturbation weight is used to adjust the proportion of the adversarial perturbation added to the foreground area and the adversarial perturbation added to the background area, mainly adding the adversarial perturbation to the foreground area, reducing the adversarial perturbation added to the background area, and improving the adversarial attack effect of the final adversarial sample image 202.

[0078] By performing the above-mentioned adversarial sample generation process on multiple original images, a set of adversarial sample images can be obtained, wherein the adversarial sample image set contains adversarial sample images corresponding to multiple original images. After obtaining a sufficient number of adversarial sample images, user A can also initiate a security assessment request for the target network model to the business server 22 through the terminal device 20. After receiving the security assessment request, the business server 22 can perform security assessment processing on the target network model. Figure 2cAs shown, the service server 22 can sequentially input adversarial sample images from the adversarial sample image set into the target network model, and then obtain the target network model's prediction results for each adversarial sample image. The service server 22 then compares each adversarial sample image's prediction results with the true label to obtain the attack results for each adversarial sample image, and thus the attack success rate of the adversarial sample image set against the target network model. If the attack success rate is greater than a security threshold, indicating that the security of the target network model is poor, the service server 22 can send an evaluation failure message 210 to the terminal device 20, such as "Model security is poor and needs optimization." If the attack success rate is less than the security threshold, indicating that the security of the target network model is good, the service server 22 can send an evaluation pass message 220 to the terminal device 20, such as "Model security is high and can be deployed and used." After user A receives the evaluation message of the target network model through the terminal device 20, he or she can choose to optimize or deploy the target network model based on the evaluation message.

[0079] Further, see Figure 3 , Figure 3 This is a flow chart of a method for generating adversarial samples provided by an embodiment of the present application. Figure 1 The business server 1000 in the corresponding embodiment is executed, and the embodiment of the present application takes the method executed by the above-mentioned business server as an example for explanation. Figure 3 As shown, the adversarial sample generation method may at least include the following steps S101 to S103:

[0080] Step S101 : obtaining an original image, and performing diversity construction processing on the original image to obtain a to-be-processed image with diversity.

[0081] Specifically, after the service server obtains the original image, it can use the Input Diversity data augmentation strategy to perform diversity construction on the original image to ensure the diversity of the final attack sample image. The original image refers to a normal image without any perturbation or distortion.

[0082] In one feasible embodiment, a service server obtains an original image, performs a diversity construction process on the original image, and obtains a diverse image to be processed. The specific process can include: obtaining the original image; then scaling the original image to obtain a scaled image. The scaled image is smaller than the original image. The scaled image size can be a pre-set size or a randomly generated size by the service server based on the original image size after obtaining the original image. For example, the scaled image size can be half the original image size, or the original image size can be input into a random function to obtain the scaled image size. During the scaling process, the scaled image size can be set based on actual circumstances and is not limited here. The service server then performs interpolation and padding on the scaled image to obtain the image to be processed. The size of the image to be processed is the same as the size of the original image. Interpolation refers to using known data to predict unknown data. Image interpolation is a process where a pixel is given and its value is predicted based on information about its surrounding pixels. Interpolation and padding are then performed on the scaled image, filling the scaled image with the interpolated pixels, thereby obtaining a processed image of the same size as the original image. There are many ways of interpolation operations, such as nearest neighbor interpolation, bilinear interpolation, and cubic convolution interpolation, etc. You can choose according to the actual situation and there is no restriction here.

[0083] Step S102: Obtain N substitute attack network models associated with the target network model to be attacked, and generate adversarial perturbations for the image to be processed through the N substitute attack network models; N is a positive integer; the substitute attack network models and the target network model have the same task objectives.

[0084] Specifically, the target network model can be an AI model with classification, object detection, semantic segmentation, instance segmentation, or other task objectives. This method uses a substitute attack network model to replace the target network model as the victim model to generate adversarial sample images. No input or output is required to the target network model during the generation of adversarial sample images. The victim model refers to the model being attacked. It is understood that the substitute attack network model and the target network model should have the same task objective. For example, if the target network model is a model for object recognition and classification, then the substitute attack network model should also be a model for object recognition and classification. However, the structure, parameters, and other information of the substitute attack network model may differ from those of the target network model. However, the recognition results of the target network model and the substitute attack network model for normal images (such as the original image mentioned above) should be the same. Assuming that both the target network model and the substitute attack network model can be used for animal recognition, and the original image is an image of a tiger, then if the original image is fed normally into the target network model, the output of the target network model can confirm that the animal in the original image is a tiger. If the original image is fed normally into the substitute attack network model, the output of the substitute attack network model should also confirm that the animal in the original image is a tiger. Although the output vectors of the target network model and the alternative attack network model are different before obtaining the recognition results due to differences in model structure and parameters, the final corresponding recognition results are the same.

[0085] Specifically, after adding some tiny disturbance noise that is difficult for the human eye to detect to the original image, the prediction result of the model will change. Such disturbance noise is called adversarial disturbance. In other words, adversarial disturbance is a small change added to the model input that can mislead the model output. The process of adding an adversarial disturbance image to mislead the target network model to output an incorrect prediction result is called an adversarial attack. When improving the robustness of the target network model to adversarial attacks, adversarial training is a commonly used method, that is, by simulating the adversarial attacks that the target network model may encounter in the real environment, generating adversarial sample images, and then attacking the target network model with the adversarial sample images, and evaluating the security of the model based on the success rate of the attack. The authenticity of the adversarial disturbance is particularly important. The embodiment of the present application proposes a box-less attack, which does not require interaction with the target network model. It directly selects an alternative attack network model associated with the target network model to generate adversarial disturbances together with the image to be processed, so as to better simulate the adversarial attacks that the target network model may encounter in the real environment.

[0086] Specifically, when the business server selects an alternative attack network model associated with the target network model, it can first obtain N alternative attack network models, where N is a positive integer. When an alternative attack network model is obtained, adversarial perturbations can be generated directly based on the image to be processed and the alternative attack network model; when multiple alternative attack network models are obtained, the idea of ensemble learning can be used to combine N alternative attack network models and generate adversarial perturbations together with the image to be processed. Among them, the idea of ensemble learning is to complete the learning task by constructing and combining multiple machine learning machines. In an embodiment of the present application, multiple alternative attack network models can be integrated before attacking. The specific implementation can be that the image to be processed is input into multiple alternative attack network models respectively, and the output vector of the alternative attack network model before the result judgment is obtained. The output vector of each alternative attack network model is fused and averaged to obtain the output vector average value, and then the output vector average value is used to calculate the loss function, thereby generating adversarial perturbations.

[0087] Step S103, adding the adversarial perturbation to the image to be processed according to the perturbation weight of the foreground area and the perturbation weight of the background area of the original image to obtain an adversarial sample image; the adversarial sample image is used to perform an adversarial attack on the target network model.

[0088] Specifically, the foreground area refers to the area occupied by the main object to be detected by the target network model in the image, and the background area refers to the area not occupied by the main object to be detected in the image. For example, Figure 2a In the original image 200 shown, the puppy is the main object to be detected, and the area it occupies is the foreground area, while the rest of the area is the background area. It is understandable that the more adversarial perturbations are added to the image to be processed, the worse the adversarial attack effect of the image to be processed. Therefore, the perturbation weight of the foreground area and the perturbation weight of the background area can be set in advance to determine the extent to which the service server adds adversarial perturbations to the image to be processed. Among them, the perturbation weight of the foreground area should be greater than the perturbation weight of the background area, because the foreground area is the main detection area of the target network model, and the adversarial perturbation added to the foreground area is the main factor affecting the misjudgment of the target network model. Adding too much adversarial perturbation to the background area will make the adversarial attack effect of the adversarial sample image worse. The specific settings can be determined according to the actual situation and are not limited here.

[0089] Optionally, one original image can generate an adversarial sample image. The service server can simultaneously perform adversarial sample generation processing on multiple original images to obtain multiple adversarial sample images. After obtaining the adversarial sample images, the service server can use them to conduct an adversarial attack on the target network model and obtain the attack success rate. The specific implementation process can be as follows: the service server inputs each adversarial sample image into the target network model, obtains the output result label of the target network model, compares the output result label of the target network model with the true label of the original image corresponding to the adversarial sample image, and if they are different, the adversarial sample image is considered to have successfully attacked. The number of successfully attacked adversarial sample images is then divided by the total number of adversarial sample images input into the target network model to obtain the attack success rate. The security level of the target network model can then be evaluated based on the attack success rate. The higher the attack success rate, the lower the security level of the target network model.

[0090] Optionally, the generation of adversarial perturbations can also be based on an unlimited attack method, using attack methods such as changing color, changing contrast, and adding special patterns.

[0091] By adopting the method provided in the embodiment of the present application, the original image can be subjected to diversity construction processing to obtain a to-be-processed image with diversity, and then N alternative attack network models associated with the target network model to be attacked are obtained, and adversarial perturbations are generated for the to-be-processed image through these N alternative attack network models; finally, the adversarial perturbations are added to the to-be-processed image according to the perturbation weights of the foreground area and the perturbation weights of the background area of the original image to obtain an adversarial sample image. Wherein, N is a positive integer; the alternative attack network model and the target network model have the same task objectives; and the adversarial sample image is used to conduct an adversarial attack on the target network model. By adopting the method provided in the present application, adversarial sample images are generated through N alternative attack network models, and adversarial attacks can be conducted on the target network model through the adversarial sample images without relying on any query of the target network model, which can improve the authenticity of the adversarial sample, more effectively simulate the attack threats that the target network model may suffer, and further improve the effectiveness of the security assessment of the target network model.

[0092] Further, see Figure 4 , Figure 4 This is a flow chart of a method for generating an anti-disturbance according to an embodiment of the present application. Figure 3 A specific implementation method of step S102 in the corresponding embodiment, the adversarial perturbation generation method adopts a gradient-based perturbation generation method, and combines momentum reset, ensemble learning and other methods to generate adversarial perturbations in an iterative manner. Figure 4As shown, the adversarial sample generation method may at least include the following steps S201 to S204:

[0093] Step S201: In the i-th training iteration, a training perturbation corresponding to the training image in the i-th training iteration is generated by the N replacement attack network models; i is a positive integer; the training image in the first training iteration is the image to be processed.

[0094] Specifically, the business server uses the image to be processed as the training image in the first training iteration, and then performs multiple rounds of training iterations. After the iteration is completed, the training perturbation generated in the last training iteration is used as the adversarial perturbation.

[0095] Specifically, in the i-th training iteration, the process of generating the training perturbations corresponding to the training images in the i-th training iteration using N surrogate attack network models can be as follows: the service server first integrates the outputs of the N surrogate attack network models based on the principle of ensemble learning. Specifically, the training images in the i-th training iteration are input into the N surrogate attack network models to obtain the result vectors output by the N surrogate attack network models. The result vectors output by the N surrogate attack network models are then averaged to obtain the average result vector. The service server then determines the loss function associated with the training images in the i-th training iteration based on the average result vector and the true label. The loss function is a non-negative real-valued function used to measure the degree of inconsistency between the model's predicted results and the true results. The smaller the loss function, the better the model's robustness. Loss functions can include absolute value loss, squared loss, exponential loss, and other functions. Different models generally use different loss functions. Therefore, the choice of loss function can be based on the specific task objectives of the target network model and is not limited here. The business server then determines the gradient of the loss function with respect to the training image in the i-th training iteration. That is, by taking the derivative of the loss function with respect to the training image in the i-th training iteration, the gradient in the i-th training iteration is obtained. The training perturbation corresponding to the training image in the i-th training iteration is then generated in the opposite direction of the gradient. The training perturbation can be generated by multiplying the gradient by the perturbation step size (learning rate). The specific algorithm for generating training perturbations based on the gradient can be the PGD (Project Gradient Descent) attack algorithm, or other attack algorithms can be used, without limitation here.

[0096] Step S202: If i is less than the iteration number threshold, the training image in the i-th training iteration is updated according to the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration to obtain the training image in the i+1-th training iteration.

[0097] Specifically, the business server adds the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration to the training image in the i-th training iteration to obtain the training image in the i+1-th training iteration.

[0098] Step S203 : updating the accumulated momentum in the i-th training iteration according to i and the training perturbation corresponding to the training image in the i-th training iteration to obtain the accumulated momentum in the (i+1)-th training iteration.

[0099] Specifically, the business server can obtain a momentum reset parameter, and then divide i by the momentum reset parameter to obtain a target remainder. If the target remainder does not meet the momentum reset condition, the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration are added together to obtain the accumulated momentum in the i+1-th training iteration; if the target remainder meets the momentum reset condition, the accumulated momentum in the i-th training iteration is reset to obtain the accumulated momentum in the i+1-th training iteration. The momentum reset condition can be whether the target remainder is zero. The reset process can be to update the accumulated momentum to an initial value, such as zero. In short, a momentum reset parameter can be set. If the accumulated number of training iterations reaches the momentum reset parameter, the accumulated momentum is updated to the initial value. If the accumulated number of training iterations does not exceed the momentum reset parameter, the training perturbation corresponding to the training image in this training iteration and the accumulated momentum in this training iteration are added together. For example, assuming i is 15 and the momentum reset parameter is 8, the target remainder obtained by dividing i by the momentum reset parameter is 7. This target remainder is not 0, indicating that 7 iterations have been accumulated and the momentum reset condition is not met. The training perturbation obtained from the current training iteration needs to be added to the accumulated momentum of this round to obtain the accumulated momentum for the next training iteration. In the next training iteration, i is 16, and the target remainder obtained by dividing i by the momentum reset parameter is 0. At this time, 8 iterations have been accumulated and the momentum reset condition is met. The accumulated momentum for the next training iteration will be reset to obtain the accumulated momentum for the 17th training iteration, that is, the accumulated momentum of the 17th training iteration is the initial value.

[0100] Step S204: In the i+1th training iteration, a training perturbation corresponding to the training image in the i+1th training iteration is generated through the N alternative attack network models; if the i+1 is equal to the iteration number threshold, the training perturbation corresponding to the training image in the i+1th training iteration is determined as the adversarial perturbation.

[0101] Specifically, during round i of training iterations, the training perturbation obtained in round i and the accumulated momentum of round i are added to the training image in round i to obtain the training image for round i+1. Then, in round i+1, the training perturbation for round i+1 is generated according to the method used to generate the training perturbation in round i. Before the start of training iterations, a threshold for the number of iterations can be set. If i+1 is equal to the threshold for the number of iterations, the training perturbation corresponding to the training image in the i+1th training iteration is determined as the adversarial perturbation. If i+1 is less than the threshold for the number of iterations, the training image for round i+2 is generated.

[0102] Optionally, in a feasible embodiment, when the training iteration ends can be determined by the loss function value, that is, when the loss function value determined in a certain round of training iteration is greater than or equal to the loss threshold, the iteration is stopped, and the training perturbation obtained in the round of training iteration is used as the adversarial perturbation. At this time, the training iteration process for generating adversarial perturbations can be: in the i-th training iteration, the training perturbation corresponding to the training image in the i-th training iteration is generated by N alternative attack network models; based on the N alternative attack network models and the training image in the i-th training iteration, the loss function value of the i-th training iteration is determined; if the loss function value of the i-th training iteration is less than the loss threshold, the training image in the i-th training iteration is updated according to the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration to obtain the training image in the i+1-th training iteration; based on the i and the i-th training iterations, the training perturbation corresponding to the training image in the i-th training iteration is updated. The training perturbation corresponding to the training image in the i-th training iteration updates the cumulative momentum in the i-th training iteration to obtain the cumulative momentum in the i+1-th training iteration; in the i+1-th training iteration, the training perturbation corresponding to the training image in the i+1-th training iteration is generated through N alternative attack network models; the loss function value of the i+1-th training iteration is determined based on the N alternative attack network models and the training image in the i+1-th training iteration; if the loss function value of the i+1-th training iteration is greater than or equal to the loss threshold, the training perturbation corresponding to the training image in the i+1-th training iteration is determined as the adversarial perturbation. Wherein, i is a positive integer. Wherein, the training image in the first training iteration is the image to be processed. The generation of the training perturbation and the update of the cumulative momentum in the above-mentioned training iteration process can be referred to the relevant descriptions in the above-mentioned steps S201-204 and will not be repeated here.

[0103] The method of the embodiment of the present application can well simulate the attacks that the target network model may suffer in reality, effectively evaluate the security of the target network model, and provide a good security warning function for AI products that use the target network model.

[0104] Further, see Figure 5 , Figure 5This is a schematic diagram of a disturbance-adding scenario provided by an embodiment of the present application. Figure 5 As shown, the service server obtains the original image 500 (for example, the above Figure 2b After the original image 200 in the corresponding embodiment is processed, the saliency segmentation process can be performed on it to obtain the foreground area position information and the background area position information. Figure 3 When the adversarial perturbation is added to the image to be processed according to the perturbation weight of the foreground region and the perturbation weight of the background region of the original image in step S103 of the corresponding embodiment, the specific implementation can be as follows: Figure 5 As shown, the service server can determine the image to be processed 501 (for example, the above Figure 2b In the corresponding embodiment, the foreground area 502 to be processed of the image to be processed 201 is obviously the main object, and therefore the foreground area 502 to be processed is the area occupied by the dog. The background area 503 to be processed of the image to be processed 501 is determined based on the background area position information, i.e., the area not occupied by the dog. Then, the service server can update the foreground area 502 to be processed of the image to be processed based on the perturbation weight of the foreground area and the adversarial perturbation, that is, multiplying the adversarial perturbation of the foreground area by the perturbation weight of the foreground area, and then adding it to the foreground area 502 to obtain a foreground adversarial image 504. Then, the background area 503 to be processed of the image to be processed is updated based on the perturbation weight of the background area and the adversarial perturbation, that is, multiplying the adversarial perturbation of the background area by the perturbation weight of the background area, and then adding it to the background area 503 to obtain a background adversarial image 505. It should be noted that the perturbation weight of the foreground area is greater than the perturbation weight of the background area, and ultimately, more adversarial perturbation is added to the foreground area than to the background area. Finally, the service server will perform image fusion on the foreground adversarial image 504 and the background adversarial image 505 to obtain the adversarial sample image 506 (i.e., the above Figure 2b The adversarial sample image 202 in the corresponding embodiment).

[0105] By using the method provided in the embodiment of the present application, different perturbation weights are set for the foreground area and the background area, wherein the foreground area occupies a larger perturbation weight, the quality of the adversarial sample image can be improved, and the attack strength of the adversarial sample image can also be increased.

[0106] For further information, see Figure 6 , Figure 6 This is a schematic diagram of the overall framework of a method for generating adversarial samples provided in an embodiment of the present application. Figure 6As shown in the figure, the overall framework of the adversarial sample generation method mainly includes two modules: a data preprocessing module 61 and an adversarial perturbation generation module 62. Among them, the data preprocessing module 61 includes two main functions: generating a sample with diversity through the InputDiversity operation, that is, the image to be processed 612; and determining the foreground area 6131 and the background area 6132 of the original image 610 through the pixel-level saliency segmentation method. Among them, the adversarial perturbation generation module 62 generates adversarial perturbations in the framework of conventional adversarial attack methods, combining model adversarial training, ensemble integration, and momentum reset strategies, and adds the adversarial perturbations to the foreground area and background area of the image to be processed 612 according to different perturbation weights to form the final attack sample, that is, the adversarial sample image 625.

[0107] Specifically, in the data preprocessing module 61, in order to ensure the diversity of the images to be processed after input into the integrated model 620, an Input Diversity data enhancement strategy can be adopted, such as Figure 6 As shown, the original image 610 can be randomly scaled to a smaller size to obtain a scaled image 611. The scaled image 611 can then be restored to its original size through interpolation and filling to obtain a processed image 612. This allows the randomness of scaling to be utilized to construct a diverse set of processed images, which significantly helps improve the generalization of the attack. Then, to accurately add adversarial perturbations to the key target area, namely the foreground region, this application introduces a pixel-level saliency segmentation method to separate the foreground region 6131 and the background region 6132 of the original image 610, obtaining foreground region position information and background region position information. Saliency segmentation refers to using the segmentation model to identify and segment areas of the original image 610 that are easily noticed from the perspective of the human eye. Segmentation is performed pixel by pixel, meaning each pixel position is assigned a pixel value representing saliency. A saliency threshold is then used to distinguish between the foreground and background. Finally, foreground pixels are combined to form the foreground region 6131, and background pixels are combined to form the background region 6132.

[0108] Specifically, the adversarial disturbance generation module 62 mainly includes the generation of adversarial disturbance, the attack of the alternative attack network model, and the addition of adversarial disturbance.

[0109] Specifically, the generation of adversarial perturbations can be performed using a "small-step iteration" approach, where perturbations are generated along the direction opposite to the gradient of the image to be processed 612 using the loss function. Specifically, in each iteration, a training image needs to be imported (the image input to the integrated model 620 in each training iteration, the training image in the first training iteration is the image to be processed 612, and the training image in each training iteration is generated by the previous training iteration starting from the second training iteration) for forward propagation, and the loss function is calculated based on the output of the integrated model 620 and the true label. Then, the gradient of the loss function corresponding to the training image is solved according to the "chain derivation" rule, and the training image is updated along the direction opposite to the gradient. The amplitude of each update is called the "perturbation step size."

[0110] Specifically, the attack of replacing the attack network model means that in the box-less scenario, the prior information of the target network model cannot be obtained, so other possible replacement attack network models are selected to replace the target model according to the task objective of the target network model, and the replacement attack network model is used as the victim model to generate the corresponding adversarial sample image 625. In order to increase the strength of the attack, multiple replacement attack network models can be obtained and integrated before attacking. Figure 6 As shown, N alternative attack network models can be obtained and integrated to obtain an integrated model 620. For example, for a classification task, the alternative attack network model will output a classification vector before the classification judgment step to represent the classification probability of each category. The integration method adopted in this solution is to average the classification vectors output by the N alternative attack network models, use the averaged classification vector to calculate the loss function, and then obtain the adversarial perturbation. In addition, in order to improve the image quality of the final adversarial sample image, a momentum reset strategy can be adopted. Under normal circumstances, the training perturbation generated in each iteration step will accumulate in the accumulated momentum, and the training image can be updated by using the accumulated momentum and gradient. After the momentum reset is adopted, the accumulated momentum is reset to zero every m iteration steps. This can avoid excessive accumulated momentum, which leads to excessive single-step training perturbation, thereby affecting the quality of the training image. Wherein, m is a positive integer.

[0111] Specifically, the addition of disturbance is mainly based on the foreground area position information and the background area position information obtained by the foreground and background segmentation of the original image 611, as well as the disturbance weight of the foreground area and the disturbance weight of the background area. For details, please refer to the above Figure 5 The description of the corresponding embodiments will not be repeated here.

[0112] The adversarial image 625 generated using this framework can be imported into the target network model for computation, and the model's security can be assessed based on the attack success rate. A lower success rate indicates a more secure model, while a higher success rate indicates a less secure model and a greater likelihood of being vulnerable to adversarial attacks.

[0113] See Figure 7 , is a schematic diagram of the structure of an adversarial sample generation device provided in an embodiment of the present application. The adversarial sample generation device can be a computer program (including program code) running on a computer device, for example, the adversarial sample generation device is an application software; the device can be used to execute the corresponding steps in the adversarial sample generation method provided in an embodiment of the present application. Figure 7 As shown, the adversarial sample generating device 1 may include: an image construction module 11 , a disturbance generating module 12 and a disturbance adding module 13 .

[0114] The image construction module 11 is used to obtain an original image and perform a diversity construction process on the original image to obtain a diverse image to be processed;

[0115] The perturbation generation module 12 is configured to obtain N substitute attack network models associated with the target network model to be attacked, and generate adversarial perturbations for the image to be processed using the N substitute attack network models; N is a positive integer; and the substitute attack network models and the target network model have the same task objective;

[0116] The perturbation adding module 13 is used to add adversarial perturbations to the image to be processed according to the perturbation weights of the foreground area and the background area of the original image to obtain adversarial sample images; the adversarial sample images are used to conduct adversarial attacks on the target network model.

[0117] The specific functional implementation of the image construction module 11, the disturbance generation module 12 and the disturbance addition module 13 can be found in Figure 3 The detailed description of steps S101 to S103 in the corresponding embodiment will not be repeated here.

[0118] Please see again Figure 7 The image construction module 11 may include: an image acquisition unit 110 , a scaling unit 111 , and an interpolation filling unit 112 .

[0119] An image acquisition unit 110 is configured to acquire an original image;

[0120] The scaling unit 111 is configured to scale the original image to obtain a scaled image; the scaled image has a size smaller than the original image;

[0121] The interpolation filling unit 112 is used to perform interpolation filling processing on the scaled image to obtain an image to be processed; the size of the image to be processed is the same as the size of the original image.

[0122] The specific functional implementation of the image acquisition unit 110, the scaling unit 111, and the interpolation filling unit 112 can be found in Figure 3 The specific description of step S101 in the corresponding embodiment will not be repeated here.

[0123] Please see again Figure 7 , may include: a first iterative processing unit 120, a first image updating unit 121, a first momentum updating unit 122, a second iterative processing unit 123 and a first disturbance determining unit 124.

[0124] The first iterative processing unit 120 is configured to generate, in the i-th training iteration, a training perturbation corresponding to the training image in the i-th training iteration using N replacement attack network models; i is a positive integer; the training image in the first training iteration is the image to be processed;

[0125] A first image updating unit 121 is configured to update the training image in the i-th training iteration according to the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration, to obtain a training image in the i+1-th training iteration, if i is less than the iteration number threshold;

[0126] A first momentum updating unit 122 is configured to update the accumulated momentum in the i-th training iteration according to the training perturbations corresponding to the training images in the i-th training iteration to obtain the accumulated momentum in the i+1-th training iteration;

[0127] The second iterative processing unit 123 is configured to generate, in the (i+1)th training iteration, a training perturbation corresponding to the training image in the (i+1)th training iteration through N replacement attack network models;

[0128] The first disturbance determination unit 124 is configured to determine the training disturbance corresponding to the training image in the (i+1)th training iteration as the adversarial disturbance if (i+1) is equal to the iteration number threshold.

[0129] The specific functional implementation of the first iterative processing unit 120, the first image updating unit 121, the first momentum updating unit 122, the second iterative processing unit 123 and the first disturbance determining unit 124 can be found in Figure 4 The detailed description of steps S201 to S204 in the corresponding embodiment will not be repeated here.

[0130] Please see again Figure 7The first iterative processing unit 120 may include: a model output subunit 1201, a vector averaging subunit 1202, a loss determination subunit 1203 and a training perturbation generation subunit 1204.

[0131] The model output subunit 1201 is used to input the training images in the i-th training iteration into N replacement attack network models respectively, and obtain the result vectors output by the N replacement attack network models respectively;

[0132] The vector averaging subunit 1202 is used to average the result vectors outputted by the N alternative attack network models to obtain an average result vector;

[0133] a loss determination subunit 1203 for determining a loss function associated with the training image in the i-th training iteration based on the average result vector and the true label;

[0134] The training perturbation generating subunit 1204 is used to determine the gradient of the loss function with respect to the training image in the i-th training iteration, and to generate the training perturbation corresponding to the training image in the i-th training iteration along the reverse direction of the gradient.

[0135] The specific functional implementation of the model output subunit 1201, the vector average subunit 1202, the loss determination subunit 1203 and the training perturbation generation subunit 1204 can be found in Figure 4 The specific description of step S201 in the corresponding embodiment will not be repeated here.

[0136] Please see again Figure 7 The first momentum updating unit 122 may include: a remainder determining subunit 1220 and a momentum updating subunit 1221 .

[0137] The remainder determination subunit 1220 is used to obtain a momentum reset parameter and divide the default number of iterations by the momentum reset parameter to obtain a target remainder;

[0138] The momentum update subunit 1221 is configured to add the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration to obtain the accumulated momentum in the i+1-th training iteration if the target remainder does not satisfy the momentum reset condition;

[0139] The momentum updating subunit 1221 is further configured to reset the accumulated momentum in the i-th training iteration if the target remainder satisfies the momentum reset condition, to obtain the accumulated momentum in the i+1-th training iteration.

[0140] The specific functional implementation of the remainder determination subunit 1220 and the momentum update subunit 1221 can be found in Figure 4The specific description of step S203 in the corresponding embodiment will not be repeated here.

[0141] Please see again Figure 7 The disturbance generation module 12 may include: a third iterative processing unit 125 , a second image updating unit 126 , a second momentum updating unit 127 , a fourth iterative processing unit 128 and a second disturbance determining unit 129 .

[0142] The third iterative processing unit 125 is configured to generate, in an i-th training iteration, a training perturbation corresponding to the training image in the i-th training iteration using N alternative attack network models, where i is a positive integer and the training image in the first training iteration is the image to be processed; and determine a loss function value for the i-th training iteration based on the N alternative attack network models and the training image in the i-th training iteration;

[0143] a second image updating unit 126 configured to update the training image in the i-th training iteration according to the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration, to obtain a training image in the (i+1)-th training iteration, if the loss function value of the i-th training iteration is less than the loss threshold;

[0144] A second momentum updating unit 127 is configured to update the accumulated momentum in the i-th training iteration according to the training perturbations corresponding to the training images in the i-th training iteration to obtain the accumulated momentum in the i+1-th training iteration;

[0145] a fourth iterative processing unit 128 configured to generate, in the (i+1)th training iteration, a training perturbation corresponding to the training image in the (i+1)th training iteration using the N alternative attack network models; and determine a loss function value for the (i+1)th training iteration based on the N alternative attack network models and the training image in the (i+1)th training iteration;

[0146] The second disturbance determination unit 129 is configured to determine the training disturbance corresponding to the training image in the (i+1)th training iteration as the adversarial disturbance if the loss function value of the (i+1)th training iteration is greater than or equal to the loss threshold.

[0147] The specific functional implementation of the third iterative processing unit 125, the second image updating unit 126, the second momentum updating unit 127, the fourth iterative processing unit 128 and the second disturbance determining unit 129 can be found in Figure 4 The optional descriptions in the corresponding embodiments will not be repeated here.

[0148] Please see again Figure 7 , the adversarial sample generating device 1 may further include: a segmentation module 14.

[0149] The segmentation module 14 is used to perform saliency segmentation processing on the original image to obtain foreground area position information and background area position information.

[0150] The disturbance adding module 13 may include: a region determining unit 130 , a foreground adding unit 131 , a background adding unit 132 , and a fusion unit 133 .

[0151] The region determining unit 130 is configured to determine a foreground region to be processed of the image to be processed according to the foreground region position information, and to determine a background region to be processed of the image to be processed according to the background region position information;

[0152] a foreground adding unit 131 for updating the foreground region to be processed of the image to be processed according to the perturbation weight of the foreground region and the adversarial perturbation to obtain a foreground adversarial image;

[0153] A background adding unit 132 is configured to update the background region to be processed of the image to be processed according to the disturbance weight of the background region and the adversarial disturbance to obtain a background adversarial image;

[0154] The fusion unit 133 is configured to fuse the foreground adversarial image and the background adversarial image to obtain an adversarial sample image.

[0155] The specific functional implementation of the segmentation module 14, the region determination unit 130, the foreground adding unit 131, the background adding unit 132 and the fusion unit 133 can be found in Figure 5 The specific description in the corresponding embodiment will not be repeated here.

[0156] Please see again Figure 7 The adversarial sample generation device 1 may further include: an attack module 15 and a security assessment module 16.

[0157] Attack module 15, used to conduct adversarial attacks on the target network model through adversarial sample images and obtain the attack success rate;

[0158] The security assessment module 16 is used to assess the security level of the target network model according to the attack success rate.

[0159] The specific functional implementation of the attack module 15 and the security assessment module 16 can be found in Figure 3 The optional descriptions in the corresponding embodiments will not be repeated here.

[0160] See Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 8As shown, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1004 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Figure 8 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application.

[0161] In such Figure 8 In the computer device 1000 shown, the network interface 1004 can provide a network communication element; the user interface 1003 is mainly used to provide an interface for user input; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0162] Acquire an original image, perform diversity construction processing on the original image, and obtain a to-be-processed image with diversity;

[0163] Obtain N alternative attack network models associated with the target network model to be attacked, and generate adversarial perturbations for the image to be processed through the N alternative attack network models; N is a positive integer; the alternative attack network models and the target network model have the same task objectives;

[0164] According to the perturbation weights of the foreground area and the background area of the original image, the adversarial perturbation is added to the image to be processed to obtain an adversarial sample image; the adversarial sample image is used to conduct an adversarial attack on the target network model.

[0165] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 3 The description of the adversarial sample generation method in the corresponding embodiment can also be performed as described above. Figure 7 The description of the adversarial sample generation device 1 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here either.

[0166] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores the computer program executed by the adversarial sample generation device 1 mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the adversarial sample generation method in any of the corresponding embodiments above. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0167] The computer-readable storage medium may be the adversarial sample generation device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0168] In addition, it should be noted that embodiments of the present application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in any of the corresponding embodiments described above.

[0169] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0170] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example in terms of network elements. Whether these network elements are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described network elements for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0171] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for generating adversarial samples, characterized in that: include: Acquiring an original image, and performing diversity construction processing on the original image to obtain a to-be-processed image with diversity; Obtain N alternative attack network models associated with the target network model to be attacked; In the i-th training iteration, a training perturbation corresponding to the training image in the i-th training iteration is generated by the N alternative attack network models; i is a positive integer; The training image in the first training iteration is the image to be processed; the training perturbation corresponding to the training image in the i-th training iteration is obtained by multiplying the gradient of the training image in the i-th training iteration by the perturbation step size; If i is less than the iteration number threshold, then updating the training image in the i-th training iteration according to the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration to obtain the training image in the i+1-th training iteration; Obtain a momentum reset parameter, and divide i by the momentum reset parameter to obtain a target remainder; If the target remainder does not satisfy the momentum reset condition, the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration are added to obtain the accumulated momentum in the i+1-th training iteration; If the target remainder satisfies the momentum reset condition, the accumulated momentum in the i-th training iteration is reset to obtain the accumulated momentum in the i+1-th training iteration; In the (i+1)th training iteration, generating a training perturbation corresponding to the training image in the (i+1)th training iteration through the N alternative attack network models; If the i+1 is equal to the iteration number threshold, the training perturbation corresponding to the training image in the i+1th training iteration is determined as the adversarial perturbation; N is a positive integer; The alternative attack network model and the target network model have the same task objective; Adding the adversarial perturbation to the image to be processed according to the perturbation weight of the foreground area and the perturbation weight of the background area of the original image to obtain an adversarial sample image; The adversarial sample image is used to perform an adversarial attack on the target network model.

2. The method according to claim 1, characterized in that The obtaining of the original image and performing diversity construction processing on the original image to obtain a diverse image to be processed includes: Get the original image; Scaling the original image to obtain a scaled image; the scaled image has a size smaller than the original image; An interpolation and filling process is performed on the scaled image to obtain an image to be processed; the size of the image to be processed is the same as the size of the original image.

3. The method according to claim 1, characterized in that Generating the training perturbation corresponding to the training image in the i-th training iteration by using the N replacement attack network models includes: Inputting the training images in the i-th training iteration into the N replacement attack network models respectively to obtain result vectors output by the N replacement attack network models respectively; Averaging the result vectors respectively output by the N alternative attack network models to obtain an average result vector; Determining a loss function associated with the training image in the i-th training iteration based on the average result vector and the true label; Determine the gradient of the loss function with respect to the training image in the i-th training iteration, and generate a training perturbation corresponding to the training image in the i-th training iteration along the reverse direction of the gradient.

4. The method according to claim 1, wherein Generating adversarial perturbations for the image to be processed by using the N alternative attack network models includes: In an i-th training iteration, generating a training perturbation corresponding to the training image in the i-th training iteration through the N alternative attack network models; i is a positive integer; the training image in the first training iteration is the image to be processed; determining a loss function value for the i-th training iteration based on the N alternative attack network models and the training image in the i-th training iteration; If the loss function value of the i-th training iteration is less than the loss threshold, then updating the training image in the i-th training iteration according to the training perturbation corresponding to the training image in the i-th training iteration and the accumulated momentum in the i-th training iteration to obtain the training image in the (i+1)-th training iteration; updating the cumulative momentum in the i-th training iteration according to i and the training perturbation corresponding to the training image in the i-th training iteration to obtain the cumulative momentum in the i+1-th training iteration; In an i+1th training iteration, generating a training perturbation corresponding to the training image in the i+1th training iteration through the N alternative attack network models; determining a loss function value for the i+1th training iteration based on the N alternative attack network models and the training image in the i+1th training iteration; If the loss function value of the (i+1)th training iteration is greater than or equal to the loss threshold, the training perturbation corresponding to the training image in the (i+1)th training iteration is determined as the adversarial perturbation.

5. The method according to claim 1, wherein Also includes: Performing saliency segmentation processing on the original image to obtain foreground area position information and background area position information; Then, the adversarial perturbation is added to the image to be processed according to the perturbation weight of the foreground area and the perturbation weight of the background area of the original image to obtain an adversarial sample image, including: Determining a foreground area to be processed of the image to be processed according to the foreground area position information, and determining a background area to be processed of the image to be processed according to the background area position information; updating the foreground region to be processed of the image to be processed according to the disturbance weight of the foreground region and the adversarial disturbance to obtain a foreground adversarial image; updating the background region to be processed of the image to be processed according to the disturbance weight of the background region and the adversarial disturbance to obtain a background adversarial image; Perform image fusion on the foreground adversarial image and the background adversarial image to obtain an adversarial sample image.

6. The method according to claim 1, characterized in that Also includes: Performing an adversarial attack on the target network model using the adversarial sample image to obtain an attack success rate; The security level of the target network model is evaluated according to the attack success rate.

7. A computer device, characterized in that: include: processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store program code, and the processor is used to call the program code to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 6.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 6 can be performed.

Images