Image processing method and device based on attack test, equipment and storage medium

Abstract

The application provides an image processing method and device based on attack testing, equipment and a storage medium. The method comprises: obtaining an image to be identified to be attacked; according to an attack intention, selecting a first object list of objects expected to be hidden in the image to be identified and a second object list of new objects expected to be generated in the image to be identified; and according to a target detector, the first object list and the second object list, iteratively processing the image to be identified through a preset attention attack strategy to obtain an attack image used to replace the image to be identified. In this way, the first object list can be used to hide the corresponding objects in the image to be identified, the second object list can be used to generate new objects in the image to be identified, various targeted objects can be flexibly simulated to generate attack pictures, and the flexibility and efficiency of attack testing can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an image processing method, apparatus, device, and storage medium based on attack testing. Background Technology

[0002] Object detection models are widely used in image detection and classification. Object detection models can generally be divided into one-stage object detectors and two-stage object detectors.

[0003] The detection method of a single-stage object detector is to directly predict the category and coordinates of different objects in the image. A typical single-stage object detection model can be the YOLO model.

[0004] The two-stage object detector works by first identifying potential target objects and then recognizing these candidate objects. A typical two-stage object detection model is the Faster-RCNN model.

[0005] Currently, object detection models can be applied to various intelligent systems such as autonomous driving and surveillance. The stability and resistance to attacks of object detection models are crucial for these systems. Currently, stability testing of object detection models typically involves deploying the intelligent system in real-world environments for practical testing. This approach is inefficient and limited in its testing scenarios. Furthermore, current attention attacks mostly target untargeted scenarios and use detector failure as a metric, making it difficult to simulate targeted attacks during stability testing of object detection models. Summary of the Invention

[0006] In view of this, the purpose of the embodiments of this application is to provide an image processing method, apparatus, device and storage medium based on attack testing, which can flexibly hide corresponding objects in the image to be identified and generate new objects, which is beneficial to improving the problem of not being able to simulate various targeted targets for attack testing.

[0007] To achieve the above technical objectives, the technical solution adopted in this application is as follows:

[0008] In a first aspect, embodiments of this application provide an image processing method based on attack testing, the method comprising:

[0009] The target image to be attacked is acquired and used for target detection.

[0010] Based on the attack intent, select a first object list that is expected to hide corresponding objects in the image to be identified and a second object list that is expected to generate new objects in the image to be identified;

[0011] Based on the target detector, the first object list, and the second object list, the image to be identified is iteratively processed using a preset attention attack strategy to obtain an attack image used to replace the image to be identified.

[0012] In conjunction with the first aspect, in some optional implementations, based on the target detector, the first object list, and the second object list, the image to be identified is iteratively processed using a preset attention attack strategy to obtain an attack image for replacing the image to be identified, including:

[0013] The first-generation result vector is obtained by identifying the image to be identified using the target detector.

[0014] Based on the first-generation result vector, the first object list, and the second object list, determine the first-generation attention heatmap corresponding to the image to be identified and the template image in the second object list;

[0015] Based on the first-generation attention heatmap, the image to be identified is updated to obtain the first-generation image;

[0016] The process is repeated to identify the k-th generation image using the target detector to obtain the (k+1)-th generation result vector, and to determine the (k+1)-th generation attention heatmap corresponding to the k-th generation image and the template image based on the (k+1)-th generation result vector, the first object list, and the second object list; and to update the k-th generation image based on the (k+1)-th generation attention heatmap to obtain the (k+1)-th generation image, and to update the value of a pre-created counter used to count the number of iterations, where k takes values ​​from 1 to T, and T is an integer greater than 1;

[0017] When the value of the iteration number k counted by the counter is updated to T, the obtained T+1th generation image is used as the attack image.

[0018] In conjunction with the first aspect, in some optional implementations, the target detector is used to identify the k-th generation image to obtain the (k+1)-th generation result vector, including:

[0019] The target detector is used to divide the k-th generation image to obtain multiple region images, wherein each region image includes at least one anchor box;

[0020] The target detector identifies each anchor frame region in each region image to obtain anchor frame data for each anchor frame, which is used as the (k+1)th generation result vector. The anchor frame data includes the coordinates, confidence level, and probability of the identified target category of the anchor frame.

[0021] In conjunction with the first aspect, in some optional implementations, determining the (k+1)th generation attention heatmap corresponding to the k-th generation image and the template image based on the (k+1)th generation result vector, the first object list, and the second object list includes:

[0022] From the k+1 generation result vector, select the anchor frame data corresponding to the target in the first object list and the second object list, and use it as the target anchor frame data;

[0023] Based on the target anchor box data, the correlation R is backpropagated from the preset deep learning model to the k-th generation image and the template image through the SGLRP algorithm, resulting in the (k+1)-th generation attention heatmap with the same size as the k-th generation image and the template image, respectively. Here, the correlation R refers to the contribution value of the neurons in the deep learning model to the prediction result.

[0024] In conjunction with the first aspect, in some optional implementations, the k-th generation image is updated based on the (k+1)-th generation attention heatmap to obtain the (k+1)-th generation image, including:

[0025] Based on the attention heatmap of the (k+1)th generation, the loss function l(x) is determined, and the formula is:

[0026]

[0027] Where h1 refers to the attention heatmap of the (k+1)th generation image, h2 refers to the attention heatmap of the template image, and h(i,j) refers to the pixel value at coordinate (i,j) in the attention heatmap.

[0028] The partial derivative g(x) is determined based on the loss function, using the following formula:

[0029]

[0030] The k-th generation image is updated based on the partial derivative, using the following formula:

[0031]

[0032] in, Refers to the image of generation k+1. The k-th generation image is referred to as , ∈ refers to the preset perturbation value, α refers to the iteration step size, and N refers to the number of pixels in the image to be identified.

[0033] In conjunction with the first aspect, in some alternative implementations, the method further includes:

[0034] The attack image is input into a target detection model based on a deep learning algorithm to obtain the target detection result output by the target detection model.

[0035] Secondly, embodiments of this application also provide an image processing apparatus based on attack testing, the apparatus comprising:

[0036] The acquisition unit is used to acquire the image to be attacked and identified.

[0037] The selection unit is used to select, according to the attack intent, a first object list in which corresponding objects are to be hidden in the image to be identified and a second object list in which new objects are to be generated in the image to be identified;

[0038] The iterative processing unit is used to iteratively process the image to be identified according to the target detector, the first object list and the second object list through a preset attention attack strategy to obtain an attack image for replacing the image to be identified.

[0039] In conjunction with the second aspect, in some optional embodiments, the iterative processing unit is further configured to:

[0040] The first-generation result vector is obtained by identifying the image to be identified using the target detector.

[0041] Based on the first-generation result vector, the first object list, and the second object list, determine the first-generation attention heatmap corresponding to the image to be identified and the template image in the second object list;

[0042] Based on the first-generation attention heatmap, the image to be identified is updated to obtain the first-generation image;

[0043] The process is repeated to identify the k-th generation image using the target detector to obtain the (k+1)-th generation result vector, and to determine the (k+1)-th generation attention heatmap corresponding to the k-th generation image and the template image based on the (k+1)-th generation result vector, the first object list, and the second object list; and to update the k-th generation image based on the (k+1)-th generation attention heatmap to obtain the (k+1)-th generation image, and to update the value of a pre-created counter used to count the number of iterations, where k takes values ​​from 1 to T, and T is an integer greater than 1;

[0044] When the value of the iteration number k counted by the counter is updated to T, the obtained T+1th generation image is used as the attack image.

[0045] In conjunction with the second aspect, in some optional embodiments, the iterative processing unit is further configured to:

[0046] The target detector is used to divide the k-th generation image to obtain multiple region images, wherein each region image includes at least one anchor box;

[0047] The target detector identifies each anchor frame region in each region image to obtain anchor frame data for each anchor frame, which is used as the (k+1)th generation result vector. The anchor frame data includes the coordinates, confidence level, and probability of the identified target category of the anchor frame.

[0048] In conjunction with the second aspect, in some optional embodiments, the iterative processing unit is further configured to:

[0049] From the k+1 generation result vector, select the anchor frame data corresponding to the target in the first object list and the second object list, and use it as the target anchor frame data;

[0050] Based on the target anchor box data, the correlation R is backpropagated from the preset deep learning model to the k-th generation image and the template image through the SGLRP algorithm, resulting in the (k+1)-th generation attention heatmap with the same size as the k-th generation image and the template image, respectively. Here, the correlation R refers to the contribution value of the neurons in the deep learning model to the prediction result.

[0051] In conjunction with the second aspect, in some optional embodiments, the iterative processing unit is further configured to:

[0052] Based on the attention heatmap of the (k+1)th generation, the loss function l(x) is determined, and the formula is:

[0053]

[0054] Where h1 refers to the attention heatmap of the (k+1)th generation image, h2 refers to the attention heatmap of the template image, and h(i,j) refers to the pixel value at coordinate (i,j) in the attention heatmap.

[0055] The partial derivative g(x) is determined based on the loss function, using the following formula:

[0056]

[0057] The k-th generation image is updated based on the partial derivative, using the following formula:

[0058]

[0059] in, Refers to the image of generation k+1. The k-th generation image is referred to as , ∈ refers to the preset perturbation value, α refers to the iteration step size, and N refers to the number of pixels in the image to be identified.

[0060] Thirdly, embodiments of this application also provide an electronic device, which includes a processor and a memory coupled to each other, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device performs the above-described method.

[0061] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods described above.

[0062] The invention employing the above technical solution has the following advantages:

[0063] In the technical solution provided in this application, based on the attack intent, a first object list is selected to hide corresponding objects in the image to be identified, and a second object list is selected to generate new objects in the image to be identified. Based on the target detector, the first object list, and the second object list, the image to be identified is iteratively processed using a preset attention attack strategy to obtain an attack image used to replace the image to be identified. Thus, the first object list can be used to hide corresponding objects in the image to be identified, and the second object list can be used to generate new objects in the image to be identified. This allows for flexible simulation of various targeted targets to generate attack images, thereby improving the flexibility and efficiency of attack testing. In other words, the attack image can be used for subsequent attack testing, which helps to enrich test scenarios and improve testing efficiency. Attached Figure Description

[0064] This application can be further illustrated by the non-limiting embodiments given in the accompanying drawings. It should be understood that the following drawings only illustrate some embodiments of this application and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained from these drawings without any inventive effort.

[0065] Figure 1 This is a flowchart illustrating the image processing method based on attack testing provided in an embodiment of this application.

[0066] Figure 2 This is a schematic diagram illustrating the implementation logic of the image processing method provided in the embodiments of this application.

[0067] Figure 3 This is a schematic diagram illustrating the process of generating a new object in an image to be identified, as provided in an embodiment of this application.

[0068] Figure 4A block diagram of an image processing apparatus based on attack testing provided in an embodiment of this application.

[0069] Icons: 200 - Image processing device; 210 - Acquisition unit; 220 - Selection unit; 230 - Iterative processing unit. Detailed Implementation

[0070] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts are referred to by the same reference numerals in the drawings or description. Implementations not shown or described in the drawings are forms known to those skilled in the art. In the description of this application, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0071] The inventors discovered that applying attention attacks to object detection typically involves techniques such as RAD (Relevance Attack on Detectors) and ASA (Attack on Separable Attention). The former focuses on the bounding boxes generated in the original image, reducing the accuracy of the object detector or even causing it to fail by suppressing the confidence of object detection within those boxes. The latter focuses on simpler physical world scenes, preventing the detector from detecting objects by suppressing foreground attention and increasing background attention.

[0072] Attention-based attacks are a method of attacking the robustness of deep learning models. Currently, most research on attention attacks focuses on image classification. The inventors have found that, because accurate classification and precise localization are both required in object detection, the application of attention attacks in image classification still presents certain complexities and difficulties. Although some methods exist for attacking object detection, these methods mostly target untargeted scenarios and use detector failure as a metric, making it difficult to simulate various targeted attacks during stability testing of object detection models. Furthermore, the targeted scenarios involving precise localization, generation, and suppression, as considered in this invention, have not yet been thoroughly explored.

[0073] In view of the above problems, the inventors of this application, after long-term research and exploration, propose the following embodiments to solve the above problems. The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0074] First Embodiment

[0075] This application provides an electronic device. The electronic device may include a processing module and a storage module. The storage module stores a computer program, which, when executed by the processing module, enables the electronic device to perform corresponding steps in the image processing method based on attack testing described below.

[0076] The electronic device can be, but is not limited to, a personal computer, the hardware of a vehicle's intelligent driving system, or other intelligent devices used for target detection. No specific limitation is made on the electronic device here.

[0077] Please refer to the reference. Figure 1 and Figure 2 This application also provides an image processing method based on attack testing, which can be simply referred to as an image processing method. This image processing method can be applied to the aforementioned electronic device, and the electronic device executes or implements the steps of the method. The image processing method may include the following steps:

[0078] Step 110: Obtain the image to be identified that is to be attacked, the image to be identified is used for target detection;

[0079] Step 120: Based on the attack intent, select a first object list that is expected to hide corresponding objects in the image to be identified and a second object list that is expected to generate new objects in the image to be identified.

[0080] Step 130: Based on the target detector, the first object list, and the second object list, the image to be identified is iteratively processed using a preset attention attack strategy to obtain an attack image used to replace the image to be identified.

[0081] The steps of the image processing method will be explained in detail below:

[0082] In step 110, the method of acquiring the image to be identified can be flexibly determined according to the actual situation. For example, the electronic device can directly acquire real-time images from a camera as the image to be identified for attack; or, the image input to the target detection model can be used as the image to be identified for attack.

[0083] Among them, object detection models refer to neural network models based on deep learning algorithms, which can be used for object detection and classification, and are well-known to those skilled in the art. For example, in the field of intelligent driving, the object detection model on the vehicle can identify targets such as traffic lights, pedestrians, and vehicles in the surrounding environment through images captured by the vehicle's onboard camera.

[0084] In step 120, the attack intent can be flexibly determined by the operator according to actual needs. For example, the attack intent may be: to hide target A in the image to be identified, so that the target detection model can never detect target A; and / or, to add a new target B in the image to be identified, so that the target detection model can always detect target B.

[0085] Understandably, the types and number of targets to be hidden in the image to be recognized can be flexibly set according to the actual situation. Furthermore, the types and number of new targets to be added to the image to be recognized can also be flexibly set according to the actual situation. For example, targets to be hidden (or generated) in the image to be recognized can be, but are not limited to, pedestrians, vehicles, etc.

[0086] Based on their attack intent, operators can select objects from a pre-created image library to form a first object list, and select new objects to be generated in the image to form a second object list. Understandably, the pre-created image library includes template images for a large number of different types of targets. If the operator does not select any objects to be hidden, the first object list is empty; if no objects to be generated are selected, the second object list is empty.

[0087] For example, if the first object list is empty, and the expected object to be generated in the second object list is target B, then the process of generating the attack image C based on the image to be identified P can be found in [reference needed]. Figure 3 .

[0088] In step 130, the object detector can be a single-stage object detector or a two-stage object detector. The single-stage object detector can be a YOLO model. The two-stage object detector can be a Faster-RCNN model.

[0089] In this embodiment, please refer to the reference again. Figure 1 and 2 Step 130 may include:

[0090] Step 131: The image to be identified is identified by the target detector to obtain the first generation result vector;

[0091] Step 132: Based on the first-generation result vector, the first object list, and the second object list, determine the first-generation attention heatmap corresponding to the image to be identified and the template image in the second object list;

[0092] Step 133: Based on the first-generation attention heatmap, update the image to be identified to obtain the first-generation image;

[0093] Step 134: Repeat the steps to identify the k-th generation image according to the target detector to obtain the (k+1)-th generation result vector, and determine the (k+1)-th generation attention heatmap corresponding to the k-th generation image and the template image according to the (k+1)-th generation result vector, the first object list and the second object list; and update the k-th generation image based on the (k+1)-th generation attention heatmap to obtain the (k+1)-th generation image, and update the value of the pre-created counter used to count the number of iterations, where k takes values ​​from 1 to T, and T is an integer greater than 1;

[0094] Step 135: When the value of the iteration number k counted by the counter is updated to T, the obtained T+1th generation image is used as the attack image. Here, the iteration number is the number of training rounds.

[0095] Step 131 may include:

[0096] The target detector divides the image to be identified into multiple region images, wherein each region image includes at least one anchor box.

[0097] The target detector identifies each anchor frame region in each region of the image to be identified, and obtains anchor frame data for each anchor frame, which is used as the first generation result vector. The anchor frame data may include, but is not limited to, the coordinates of the anchor frame, the width and height of the anchor frame, the confidence level, and the probability of the identified target category.

[0098] Step 132 may include:

[0099] From the first-generation result vector, select the anchor box data corresponding to the target in the first object list and the second object list as the target anchor box data (for example, assuming that the target category in the first-generation result vector includes pedestrians but not vehicles, the first object list includes pedestrians and the second object list includes vehicles, then the anchor box data of the pedestrian in the image to be identified will be used as the target anchor box data, where pedestrians are the targets to be hidden and vehicles are the targets to be regenerated).

[0100] Based on the target anchor box data, the correlation R is backpropagated from the preset deep learning model to the image to be identified and the template image through the SGLRP (Softmax-Gradient Layer-wise RelevancePropagation) algorithm, resulting in the first-generation attention heatmap with the same size as the image to be identified and the template image, respectively. Here, the correlation R refers to the contribution value of the neurons in the deep learning model to the prediction result.

[0101] The SGLRP algorithm is a classic algorithm for calculating heatmaps, and the implementation process of determining attention heatmaps will not be described in detail here.

[0102] Step 133 may include:

[0103] Based on the first-generation attention heatmap, the loss function l(x) is determined, and the formula is:

[0104]

[0105] At this point, h1 refers to the attention heatmap of the first generation image, h2 refers to the attention heatmap of the template image, and h(i,j) refers to the pixel value at coordinate (i,j) in the attention heatmap.

[0106] The partial derivative g(x) is determined based on the loss function, using the following formula:

[0107]

[0108] The image to be identified is updated based on the partial derivative, using the following formula:

[0109]

[0110] At this point, k takes the value of 0. Refers to the first generation of images. The image to be identified is denoted by , ∈ refers to the preset perturbation value, α refers to the iteration step size, and N refers to the number of pixels in the image to be identified.

[0111] In this embodiment, in step 134, the target detector identifies the k-th generation image to obtain the (k+1)-th generation result vector (the implementation process is similar to step 131), including:

[0112] The target detector is used to divide the k-th generation image to obtain multiple region images, wherein each region image includes at least one anchor box;

[0113] The target detector identifies each anchor frame region in each region image to obtain anchor frame data for each anchor frame, which is used as the (k+1)th generation result vector. The anchor frame data includes the coordinates, confidence level, and probability of the identified target category of the anchor frame.

[0114] In this embodiment, in step 134, based on the (k+1)th generation result vector, the first object list, and the second object list, the (k+1)th generation attention heatmap corresponding to the k-th generation image and the template image is determined (the implementation process is similar to step 132), including:

[0115] From the k+1 generation result vector, select the anchor frame data corresponding to the target in the first object list and the second object list, and use it as the target anchor frame data;

[0116] Based on the target anchor box data, the correlation R is backpropagated from the preset deep learning model to the k-th generation image and the template image through the SGLRP algorithm, resulting in the (k+1)-th generation attention heatmap with the same size as the k-th generation image and the template image, respectively. Here, the correlation R refers to the contribution value of the neurons in the deep learning model to the prediction result.

[0117] In this embodiment, in step 134, the k-th generation image is updated based on the (k+1)-th generation attention heatmap to obtain the (k+1)-th generation image (the implementation process is similar to step 133), including:

[0118] Based on the attention heatmap of the (k+1)th generation, the loss function l(x) is determined, and the formula is:

[0119]

[0120] Where h1 refers to the attention heatmap of the (k+1)th generation image, h2 refers to the attention heatmap of the template image, and h(i,j) refers to the pixel value at coordinate (i,j) in the attention heatmap.

[0121] The partial derivative g(x) is determined based on the loss function, using the following formula:

[0122]

[0123] The k-th generation image is updated based on the partial derivative, using the following formula:

[0124]

[0125] in, Refers to the image of generation k+1. The k-th generation image is referred to as , ∈ refers to the preset perturbation value, α refers to the iteration step size, and N refers to the number of pixels in the image to be identified.

[0126] In this embodiment, attention-based attacks refer to attacks in image classification problems that subtly alter hotspot regions in an image that are sensitive to deep learning models, inducing the model to make classification errors. Operators can use this attack intent to generate a new target of a specified size at a designated location in the image to be identified. That is, a targeted attack that generates a specific target at a specified location in the image can be used to test the security and robustness of the target detection model under attention-based attacks.

[0127] As an optional implementation, the method may further include:

[0128] Step 140: Input the attack image into a target detection model based on a deep learning algorithm to obtain the target detection result output by the target detection model.

[0129] Understandably, based on the target detection results, the stability of the target detection model can be judged by combining the attack intent. For example, if the attack intent is to hide target A in the image to be identified, and target A still exists in the target detection results of the image to be identified, it indicates that the target detection model has high stability; if target A does not exist in the target detection results, it indicates that the target detection model has low stability, and engineers need to improve the model to enhance its resistance to attacks and its stability.

[0130] It should be noted that the target detector in step 130 and the target detection model in step 140 can refer to the same detector or detection model, or they can refer to two different detectors.

[0131] To facilitate understanding, the implementation process of the method will be illustrated with examples below:

[0132] First, input the image x to be attacked. Then, input the list of real labels y = [o1, ..., o2] on image x that you want to eventually remove. n ], where o i =[c i ,x i ,y i ,w i ,h i Describe the category, top-left corner coordinates, width, and height of an object in the image; y represents the list of target objects that you want to ultimately display in the image x. ′ =[o ′ 1,…,o ′ m Train an object detector f such that, for a given perturbation range ∈, it can compute a small perturbation Δx satisfying ‖Δx‖≤∈, such that x+Δx can prevent the object detection model from recognizing y or misidentify y′.

[0133] Suppose that the image x to be identified contains only three targets: A, B, and C (i.e., a single-stage object detector can only detect targets A, B, and C). If the attack intent is to hide some targets, the following operations can be performed:

[0134] An attacker can choose target A in the attack graph, hoping that the target detection model will fail to detect this target. In this case, the attacker can add information related to target A to the first object list; while targets B and C in the detection box, being irrelevant to the attack intent, do not need to be added to the list.

[0135] If the attack intent is to add additional targets, the following operations can be performed:

[0136] Take a typical background image P as the image to be identified, assuming that there is no target A in the background image P. If an attacker wants to mislead a single-stage object detection model to identify a target A in the background image P, they can do so by selecting a template image corresponding to target A and then moving the template image of target A to an appropriate position in the background image P.

[0137] Attention-based attack mechanisms generate perturbations on the original image, simultaneously suppressing the original target and generating a specified target at a designated location. The main advantages of attention-based attack mechanisms are their superior transferability and interpretability through the attention mechanism of neural network structures; and their ability to generate attack targets more accurately than other attack methods. Thus, for the image to be attacked, the attack can simultaneously hide some of the original target and implant other targets. Attackers can add small perturbations to induce the model to misidentify a given object in a given region with high accuracy, while failing to recognize some of the original objects.

[0138] In autonomous driving scenarios, the method provided in this invention can serve as a safety testing tool for autonomous driving systems. The object detector is a crucial part of the perception module in an autonomous driving system, responsible for detecting and classifying objects in front of the autonomous vehicle (such as surrounding vehicles, behaviors, road signs, etc.). The method in this invention can mislead the object detection model into detecting a red light as a green light, interfering with the intelligent driving decision-making process, and can also mislead the object detection model into failing to recognize pedestrians, other vehicles, etc., in the lane, thereby determining the stability of the object detection model. These dangerous scenarios can all be tested and detected using the method in this invention before the mass production and market launch of autonomous driving systems, preventing unstable autonomous driving systems from causing excessively serious traffic hazards. Through attack detection, engineers can better understand the defects of the object detection model and promptly fix model problems, further strengthening the security and stability of the autonomous driving object detection model.

[0139] Second Embodiment

[0140] Please refer to Figure 4This application also provides an image processing apparatus 200 based on attack testing. The image processing apparatus 200 includes at least one software functional module that can be stored in a storage module or embedded in an electronic device's operating system (OS) in the form of software or firmware. The processing module is used to execute executable modules stored in the storage module, such as the software functional modules and computer programs included in the image processing apparatus 200.

[0141] The image processing apparatus 200 may include an acquisition unit 210, a selection unit 220, and an iterative processing unit 230. The functions of each unit may be as follows:

[0142] Acquisition unit 210 is used to acquire the image to be attacked and identified;

[0143] Selection unit 220 is used to select, according to the attack intent, a first object list that is expected to hide corresponding objects in the image to be identified and a second object list that is expected to generate new objects in the image to be identified;

[0144] The iterative processing unit 230 is used to iteratively process the image to be identified according to the target detector, the first object list and the second object list through a preset attention attack strategy to obtain an attack image for replacing the image to be identified.

[0145] Optionally, the iterative processing unit 230 is further configured to:

[0146] The first-generation result vector is obtained by identifying the image to be identified using the target detector.

[0147] Based on the first-generation result vector, the first object list, and the second object list, determine the first-generation attention heatmap corresponding to the image to be identified and the template image in the second object list;

[0148] Based on the first-generation attention heatmap, the image to be identified is updated to obtain the first-generation image;

[0149] The process is repeated to identify the k-th generation image using the target detector to obtain the (k+1)-th generation result vector, and to determine the (k+1)-th generation attention heatmap corresponding to the k-th generation image and the template image based on the (k+1)-th generation result vector, the first object list, and the second object list; and to update the k-th generation image based on the (k+1)-th generation attention heatmap to obtain the (k+1)-th generation image, and to update the value of a pre-created counter used to count the number of iterations, where k takes values ​​from 1 to T, and T is an integer greater than 1;

[0150] When the value of the iteration number k counted by the counter is updated to T, the obtained T+1th generation image is used as the attack image.

[0151] Optionally, the iterative processing unit 230 is further configured to:

[0152] The target detector is used to divide the k-th generation image to obtain multiple region images, wherein each region image includes at least one anchor box;

[0153] The target detector identifies each anchor frame region in each region image to obtain anchor frame data for each anchor frame, which is used as the (k+1)th generation result vector. The anchor frame data includes the coordinates, confidence level, and probability of the identified target category of the anchor frame.

[0154] Optionally, the iterative processing unit 230 is further configured to:

[0155] From the k+1 generation result vector, select the anchor frame data corresponding to the target in the first object list and the second object list, and use it as the target anchor frame data;

[0156] Based on the target anchor box data, the correlation R is backpropagated from the preset deep learning model to the k-th generation image and the template image through the SGLRP algorithm, resulting in the (k+1)-th generation attention heatmap with the same size as the k-th generation image and the template image, respectively. Here, the correlation R refers to the contribution value of the neurons in the deep learning model to the prediction result.

[0157] Optionally, the iterative processing unit 230 is further configured to:

[0158] Based on the attention heatmap of the (k+1)th generation, the loss function l(x) is determined, and the formula is:

[0159]

[0160] Where h1 refers to the attention heatmap of the (k+1)th generation image, h2 refers to the attention heatmap of the template image, and h(i,j) refers to the pixel value at coordinate (i,j) in the attention heatmap.

[0161] The partial derivative g(x) is determined based on the loss function, using the following formula:

[0162]

[0163] The k-th generation image is updated based on the partial derivative, using the following formula:

[0164]

[0165] in, Refers to the image of generation k+1. The k-th generation image is referred to as , ∈ refers to the preset perturbation value, α refers to the iteration step size, and N refers to the number of pixels in the image to be identified.

[0166] Optionally, the image processing apparatus 200 may further include a detection unit. The detection unit is used to input the attack image into a target detection model based on a deep learning algorithm to obtain a target detection result output by the target detection model.

[0167] It should be noted that the processing module in the electronic device can be an integrated circuit chip with signal processing capabilities. The aforementioned processing module can be a general-purpose processor. For example, the processor can be a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0168] The storage module can be, but is not limited to, random access memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, etc. In this embodiment, the storage module can be used to store the image to be identified, the attack image, the preset attention attack strategy, etc. Of course, the storage module can also be used to store a program, which the processing module executes after receiving the execution instruction.

[0169] Furthermore, those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic device described above can be referred to the corresponding steps in the aforementioned method, and will not be elaborated further here.

[0170] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the image processing method based on attack testing as described in the above embodiments.

[0171] Based on the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by hardware or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, electronic device, or network device, etc.) to execute the methods described in the various implementation scenarios of this application.

[0172] In the solution provided by this invention, in an image classification task, after obtaining the class confidence vector output by the deep learning model, the attacker calculates the heatmap of the sensitive region in the original image that enables the model to make the correct decision through backpropagation. This region is the area where the deep learning model's "attention" is most concentrated. Based on the calculated heatmap, the attacker alters the original image and attempts to suppress the attention of the correct class and increase the attention of the incorrect class, inducing the model to "shift" its attention to the incorrect class, causing the model to make an incorrect judgment. Since the "attention" regions that deep learning models focus on during decision-making have strong similarities, attention-based attack methods are often used for good generalization and transfer capabilities, while also possessing good interpretability.

[0173] In the embodiments provided in this application, it should be understood that the disclosed apparatus, systems, and methods can also be implemented in other ways. The apparatus, systems, and methods embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0174] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method based on attack testing, characterized in that, The method includes: The target image to be attacked is acquired and used for target detection. Based on the attack intent, select a first object list that is expected to hide corresponding objects in the image to be identified and a second object list that is expected to generate new objects in the image to be identified; Based on the target detector, the first object list, and the second object list, the image to be identified is iteratively processed using a preset attention attack strategy to obtain an attack image used to replace the image to be identified. Specifically, based on the target detector, the first object list, and the second object list, the image to be identified is iteratively processed using a preset attention attack strategy to obtain an attack image for replacing the image to be identified, including: The first-generation result vector is obtained by identifying the image to be identified using the target detector. Based on the first-generation result vector, the first object list, and the second object list, determine the first-generation attention heatmap corresponding to the image to be identified and the template image in the second object list; Based on the first-generation attention heatmap, the image to be identified is updated to obtain the first-generation image; The process is repeated to identify the k-th generation image using the target detector to obtain the (k+1)-th generation result vector, and to determine the (k+1)-th generation attention heatmap corresponding to the k-th generation image and the template image based on the (k+1)-th generation result vector, the first object list, and the second object list; and to update the k-th generation image based on the (k+1)-th generation attention heatmap to obtain the (k+1)-th generation image, and to update the value of a pre-created counter used to count the number of iterations, where k takes values ​​from 1 to T, and T is an integer greater than 1; When the value of the iteration number k counted by the counter is updated to T, the obtained T+1th generation image is taken as the attack image; Specifically, based on the (k+1)th generation attention heatmap, the k-th generation image is updated to obtain the (k+1)th generation image, including: Based on the attention heatmap of the (k+1)th generation, determine the loss function. The formula is: ； in, The attention heatmap of the (k+1)th generation image. The attention heatmap of the template image. Coordinates in the attention heatmap Pixel values; Determine the partial derivative based on the loss function. The formula is: ； The k-th generation image is updated based on the partial derivative, using the following formula: ； in, Refers to the image of generation k+1. Refers to the image of the kth generation. Refers to the preset disturbance value. Refers to the iteration step size. This refers to the number of pixels in the image to be identified.

2. The method according to claim 1, characterized in that, Based on the target detector, the k-th generation image is identified to obtain the (k+1)-th generation result vector, which includes: The target detector is used to divide the k-th generation image to obtain multiple region images, wherein each region image includes at least one anchor box; The target detector identifies each anchor frame region in each region image to obtain anchor frame data for each anchor frame, which is used as the (k+1)th generation result vector. The anchor frame data includes the coordinates, confidence level, and probability of the identified target category of the anchor frame.

3. The method according to claim 2, characterized in that, Based on the (k+1)th generation result vector, the first object list, and the second object list, determine the (k+1)th generation attention heatmap corresponding to the k-th generation image and the template image, including: From the k+1 generation result vector, select the anchor frame data corresponding to the target in the first object list and the second object list, and use it as the target anchor frame data; Based on the target anchor box data, the correlation R is backpropagated from the preset deep learning model to the k-th generation image and the template image through the SGLRP algorithm, resulting in the (k+1)-th generation attention heatmap with the same size as the k-th generation image and the template image, respectively. Here, the correlation R refers to the contribution value of the neurons in the deep learning model to the prediction result.

4. The method according to claim 1, characterized in that, The method further includes: The attack image is input into a target detection model based on a deep learning algorithm to obtain the target detection result output by the target detection model.

5. An image processing apparatus based on attack testing, characterized in that, The device includes: The acquisition unit is used to acquire the image to be attacked and identified. The selection unit is used to select, according to the attack intent, a first object list in which corresponding objects are to be hidden in the image to be identified and a second object list in which new objects are to be generated in the image to be identified; An iterative processing unit is used to iteratively process the image to be identified according to the target detector, the first object list and the second object list through a preset attention attack strategy to obtain an attack image for replacing the image to be identified. The iterative processing unit is also used for: The first-generation result vector is obtained by identifying the image to be identified using the target detector. Based on the first-generation result vector, the first object list, and the second object list, determine the first-generation attention heatmap corresponding to the image to be identified and the template image in the second object list; Based on the first-generation attention heatmap, the image to be identified is updated to obtain the first-generation image; The process is repeated to identify the k-th generation image using the target detector to obtain the (k+1)-th generation result vector, and to determine the (k+1)-th generation attention heatmap corresponding to the k-th generation image and the template image based on the (k+1)-th generation result vector, the first object list, and the second object list; and to update the k-th generation image based on the (k+1)-th generation attention heatmap to obtain the (k+1)-th generation image, and to update the value of a pre-created counter used to count the number of iterations, where k takes values ​​from 1 to T, and T is an integer greater than 1; When the value of the iteration number k counted by the counter is updated to T, the obtained T+1th generation image is taken as the attack image; The iterative processing unit is also used for: Based on the attention heatmap of the (k+1)th generation, determine the loss function. The formula is: ； in, The attention heatmap of the (k+1)th generation image. The attention heatmap of the template image. Coordinates in the attention heatmap Pixel values; Determine the partial derivative based on the loss function. The formula is: ； The k-th generation image is updated based on the partial derivative, using the following formula: ； in, Refers to the image of generation k+1. Refers to the image of the kth generation. Refers to the preset disturbance value. Refers to the iteration step size. This refers to the number of pixels in the image to be identified.

6. The apparatus according to claim 5, characterized in that, The iterative processing unit is also used for: The target detector is used to divide the k-th generation image to obtain multiple region images, wherein each region image includes at least one anchor box; The target detector identifies each anchor frame region in each region image to obtain anchor frame data for each anchor frame, which is used as the (k+1)th generation result vector. The anchor frame data includes the coordinates, confidence level, and probability of the identified target category of the anchor frame.

7. The apparatus according to claim 6, characterized in that, The iterative processing unit is also used for: From the k+1 generation result vector, select the anchor frame data corresponding to the target in the first object list and the second object list, and use it as the target anchor frame data; Based on the target anchor box data, the correlation R is backpropagated from the preset deep learning model to the k-th generation image and the template image through the SGLRP algorithm, resulting in the (k+1)-th generation attention heatmap with the same size as the k-th generation image and the template image, respectively. Here, the correlation R refers to the contribution value of the neurons in the deep learning model to the prediction result.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory coupled together, the memory storing a computer program that, when executed by the processor, causes the electronic device to perform the method as described in any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 4.

Images