Adversarial sample image generation method, training method and target detection method

Abstract

The present disclosure provides an adversarial sample image generation method, a target detection model training method, a target detection method, an apparatus, an electronic device, a storage medium and a computer program product, relates to the technical field of artificial intelligence, and in particular to the technical fields of image processing, target detection and deep learning. The specific implementation scheme of the adversarial sample image generation method is as follows: a window mask is used to slide a window in a target object region of an initial sample image to obtain at least one intermediate sample image; at least one perturbation region is determined according to a target object detection result of the at least one intermediate sample image and a corresponding sliding window; and a target adversarial sample image is generated according to an intermediate sample image having a perturbation region in the at least one intermediate sample image.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the fields of image processing, object detection, and deep learning technology. Specifically, it relates to an adversarial example image generation method, an object detection model training method, an object detection method, an apparatus, an electronic device, a storage medium, and a computer program product. Background Technology

[0002] With the development of artificial intelligence technology, object detection technology has become increasingly mature. Higher requirements are being placed on the accuracy and performance of object detection models used for object detection. Summary of the Invention

[0003] This disclosure provides a method for generating adversarial sample images, a method for training an object detection model, an object detection method, an apparatus, an electronic device, a storage medium, and a computer program product.

[0004] According to one aspect of this disclosure, an adversarial sample image generation method is provided, comprising: using a window mask to slide a window over a target object region of an initial sample image to obtain at least one intermediate sample image; determining at least one perturbation region based on the target object detection result of the at least one intermediate sample image and the corresponding sliding window; and generating a target adversarial sample image based on the intermediate sample image with the perturbation region in the at least one intermediate sample image.

[0005] According to another aspect of this disclosure, a method for training an object detection model is provided, comprising: inputting multiple object detection sample images into an initial object detection model to obtain object detection results, wherein the multiple object detection sample images include at least one object adversarial sample image, the object adversarial sample image being obtained using an adversarial sample image generation method according to an embodiment of this disclosure; and adjusting the model parameters of the initial object detection model according to the value of a model training loss function to obtain an object detection model, wherein the model training loss function is related to the object detection results and labels of the object detection sample images.

[0006] According to another aspect of this disclosure, a target detection method is provided, comprising: inputting an image to be detected into a target detection model to obtain a target object detection result, wherein the target detection model is obtained using a training method for a target detection model according to an embodiment of this disclosure.

[0007] According to another aspect of this disclosure, an adversarial example image generation apparatus is provided, comprising: an intermediate sample image determination module, a perturbation region determination module, and a target adversarial example image generation module. The intermediate sample image determination module is used to perform sliding windowing on a target object region of an initial sample image using a window mask to obtain at least one intermediate sample image. The perturbation region determination module is used to determine at least one perturbation region based on the target object detection result for the at least one intermediate sample image and the corresponding sliding window. The target adversarial example image generation module is used to generate a target adversarial example image based on the intermediate sample image with the perturbation region from the at least one intermediate sample image.

[0008] According to another aspect of this disclosure, a training apparatus for an object detection model is provided, comprising: an object detection result determination module and a model parameter adjustment module. The object detection result determination module is used to input multiple object detection sample images into an initial object detection model to obtain object detection results, wherein the multiple object detection sample images include at least one adversarial sample image, which is obtained using the adversarial sample image generation apparatus of the embodiments of this disclosure. The model parameter adjustment module is used to adjust the model parameters of the initial object detection model according to the value of the model training loss function to obtain an object detection model, wherein the model training loss function is related to the object detection results and labels of the object detection sample images.

[0009] According to another aspect of this disclosure, an object detection apparatus is provided, comprising: an object detection module for inputting an image to be detected into an object detection model to obtain an object detection result, wherein the object detection model is obtained using a training apparatus for an object detection model according to an embodiment of this disclosure.

[0010] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the adversarial example image generation method, the target detection model training method, and the target detection method of the embodiments of this disclosure.

[0011] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the adversarial example image generation method, the target detection model training method, and the target detection method of embodiments of this disclosure.

[0012] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the adversarial example image generation method, the target detection model training method, and the target detection method of the embodiments of this disclosure.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0015] Figure 1 The system architecture of the adversarial example image generation method, the target detection model training method, the target detection method, and the apparatus suitable for embodiments of this disclosure is illustrated by way of example.

[0016] Figure 2 An exemplary flowchart of an adversarial sample image generation method according to an embodiment of the present disclosure is shown;

[0017] Figure 3 A schematic diagram of an adversarial example image generation method according to another embodiment of the present disclosure is illustrated;

[0018] Figure 4 A schematic diagram illustrating the determination of at least one disturbance region according to an embodiment of the present disclosure is shown.

[0019] Figure 5 A schematic diagram illustrating the generation of a target adversarial sample image according to an embodiment of the present disclosure is shown.

[0020] Figure 6 A flowchart illustrating a training method for an object detection model according to an embodiment of the present disclosure is provided.

[0021] Figure 7 An exemplary flowchart of a target detection method according to an embodiment of the present disclosure is shown;

[0022] Figure 8 A block diagram of an adversarial sample image generation apparatus according to an embodiment of the present disclosure is shown as an example;

[0023] Figure 9 An exemplary block diagram of a training apparatus for an object detection model according to an embodiment of the present disclosure is shown;

[0024] Figure 10 A block diagram of a target detection apparatus according to an embodiment of the present disclosure is shown as an example; and

[0025] Figure 11 A block diagram of an electronic device that can implement the adversarial example image generation method, object detection model training method, and object detection method according to embodiments of the present disclosure is shown as an example. Detailed Implementation

[0026] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0027] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0028] The system architecture of the adversarial example image generation method, the target detection model training method, the target detection method, and the corresponding device suitable for embodiments of this disclosure is described below.

[0029] Figure 1 An exemplary system architecture suitable for embodiments of this disclosure is illustrated. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other environments or scenarios.

[0030] like Figure 1 As shown, in this embodiment of the present disclosure, the system architecture 100 may include: a terminal 101 for generating adversarial sample images of targets, a terminal 102 for training a target detection model, and a terminal 103 for detecting target objects.

[0031] In this embodiment of the disclosure, terminal 101 can be used to execute an adversarial example image generation method to obtain a target adversarial example image. Terminal 102 can execute a training method for a corresponding target detection model based on the target adversarial example image generated by terminal 101 to achieve corresponding model training. Terminal 103 can perform target object detection based on the target detection model obtained by terminal 102.

[0032] It should be noted that the generation of adversarial sample images and the training of the target detection model can be carried out on the same terminal or on different terminals.

[0033] Terminal 101, Terminal 102 and Terminal 103 can be servers or server clusters.

[0034] It should be understood that Figure 1 The number of terminals 101, 102, and 103 is merely illustrative. Depending on implementation needs, there can be any number of terminals 101, 102, and 103.

[0035] With the development of artificial intelligence technology, object detection technology has become increasingly mature. In fields such as autonomous driving, for object detection models used for object detection, due to safety requirements, it is necessary to add physical world attacks to the models, interfere with sample images and generate adversarial examples, test the effect of making the target object disappear or misclassify the target object, verify the security of the object detection model, and lay the foundation for improving the robustness of the object detection model in the real world.

[0036] Figure 2 A flowchart of an adversarial sample image generation method according to an embodiment of the present disclosure is shown as an example.

[0037] like Figure 2 As shown, an adversarial sample image generation method 200 according to an embodiment of the present disclosure includes operations S210 to S230.

[0038] In operation S210, a sliding window is used to perform a sliding window operation on the target object region of the initial sample image to obtain at least one intermediate sample image.

[0039] A window mask can be understood as a mask of the shape of a window. Window masks can be used to obscure a portion of the initial sample image.

[0040] The initial sample image contains the target object region, which can be understood as the area where the target object is located. The target object region may include, for example, a detection box containing the target object. The following example uses a vehicle as the target object, which can be applied to scenarios such as autonomous driving. It should be understood that the target object is not limited to vehicles; it can be any object.

[0041] When using a window mask to slide across the target object region of the initial sample image, before the first slide, the image formed by the window mask obscuring the initial sample image can be used as an intermediate sample image; after each slide, the image formed by the window mask obscuring the initial sample image can be used as an intermediate sample image.

[0042] In operation S220, at least one perturbation region is determined based on the target object detection result for at least one intermediate sample image and the corresponding sliding window.

[0043] The target object detection result can be understood as the detection result for the target object.

[0044] For example, the target object detection result may include the category confidence of the target object.

[0045] For example, the target object detection result may also include the target object region.

[0046] The perturbation region can be understood as the area containing the window mask that affects the target object detection result. For example, the target object region can be used as the target object detection result of the initial sample image, where the target object is displayed. When the window mask occludes the target object region, it is expected that the target object can still be identified when performing target object detection on the intermediate sample image. However, in reality, because the window mask of the intermediate sample image occludes at least part of the target object region, target object detection on the intermediate sample image may result in errors such as incorrect identification of the target object's category or location. In this case, the intermediate sample image can be considered to achieve an attack effect on target object detection, and the corresponding window mask region can be considered as a perturbation region.

[0047] In operation S230, a target adversarial sample image is generated based on an intermediate sample image with a perturbation region in at least one intermediate sample image.

[0048] Each intermediate sample image has a window mask region. In some intermediate sample images, the window mask region does not affect the target object detection result, while in others, the window mask region does affect the target object detection result, i.e., it has a perturbation region. According to embodiments of this disclosure, intermediate sample images with perturbation regions can be used to generate adversarial target images.

[0049] According to the adversarial sample image generation method of this disclosure, by using a window mask to slide a window over the target object region of an initial sample image, the target object detection result of the intermediate sample image formed by each slide of the sliding window can be used to select the location of the perturbation region. This allows for the determination of perturbation regions with attack effects on target object detection, and further, the determination of perturbation regions with even better attack effects. Correspondingly, adversarial sample images with attack effects on target object detection and those with even better attack effects can be obtained.

[0050] For example, target detection models can be trained using adversarial sample images of targets to improve their performance.

[0051] Figure 3 A schematic diagram of an adversarial sample image generation method 300 according to another embodiment of the present disclosure is shown.

[0052] like Figure 3As shown, a specific example of operation S310 is illustrated, which uses a window mask to slide a window over the target object region of the initial sample image to obtain at least one intermediate sample image. Figure 3 An example is shown, comprising the target object region TZ and a total of n intermediate sample images, from intermediate sample image 302-1 to intermediate sample image 302-n. The region containing the window mask corresponding to each intermediate sample image is also schematically shown. For example, intermediate sample image 302-1 corresponds to the window mask region w1.

[0053] For example, a window mask can be used to slide a window over the target object region TZ of the initial sample image 301 according to sliding window parameters, which may include at least one of the size of the sliding window and the sliding step size.

[0054] For example, the window mask can be any pixel value between 0 and 255.

[0055] For example, at least one of the sliding window size and the sliding step size can be adjusted. The sliding step size can affect the location of the perturbation region, and the sliding window size can affect the size of the perturbation region. When the sliding window size and sliding step size are adjustable, a perturbation region with more accurate size and location can be obtained. Target adversarial examples with this perturbation region can, for example, accurately and efficiently attack target detection models.

[0056] Taking object detection models as an example, inputting adversarial example images of targets into the model yields the category confidence score of the target object. This category confidence score can be used to measure the attack effectiveness of the adversarial example images on the object detection model.

[0057] When multiple adversarial sample images of targets have the same attack effect, and the size of the perturbation region is the same as the size of the sliding window, the smaller the size of the sliding window, the smaller the corresponding size of the perturbation region. Target adversarial sample images with smaller perturbation regions can attack target detection models more accurately and effectively.

[0058] When the size of the perturbation region in any number of adversarial examples is the same as the size of the sliding window, a larger sliding window can identify a larger perturbation region. Adversarial example images with larger perturbation regions can attack target detection models relatively more effectively.

[0059] like Figure 3 As shown, a specific example of operation S320 determining at least one perturbation region based on the target object detection result for at least one intermediate sample image and the corresponding sliding window is also schematically illustrated. Figure 3In the example, perturbation regions da-1 and da-2 are schematically shown, and intermediate sample image 303-1 with perturbation region da-1 and intermediate sample image 303-2 with perturbation region da-2 are also schematically shown.

[0060] Figure 4 A schematic diagram illustrating the determination of at least one disturbance region according to an embodiment of the present disclosure is shown.

[0061] like Figure 4 As shown, for example, the specific example of determining at least one perturbation region based on the target object detection result and the corresponding sliding window for at least one intermediate sample image in operation S420 can be implemented using the following embodiment.

[0062] In operation S421, target detection is performed on each intermediate sample image to obtain the target object detection result for each intermediate sample image.

[0063] exist Figure 4 The example schematically illustrates m intermediate sample images, from intermediate sample image 401-1 to intermediate sample image 401-m. Each intermediate sample image corresponds to a target object region TZ displaying the target object A, and each intermediate sample image corresponds to a specific window mask. For example, intermediate sample image 401-1 corresponds to window mask w1. Target detection can be performed on each intermediate sample image to obtain the corresponding target object detection result; for example, intermediate sample image 401-1 corresponds to target object detection result 402-1.

[0064] In operation S422, the target intermediate sample image is determined based on the target object detection results of the intermediate sample image.

[0065] For example, the target object detection result may include the category confidence of the target object.

[0066] like Figure 4As shown, for example, the target object region TZ in intermediate sample image 401-1 corresponds to the category of target object A. The target object detection result 402-1 in intermediate sample image 401-1 indicates that the category confidence (cc-A) of target object A is 30%, the category confidence (cc-B) of object B is 60%, and the category confidence (cc-C) of object C is 10%. This means that the target object detection result in intermediate sample image 401-1 indicates that the target object region TZ is object category B (object category B is determined based on the object category with the highest category confidence value). However, intermediate sample image 401-1 actually displays the category of target object A. It can be determined that at least due to the window mask w1 partially obscuring the target object region TZ, the target object detection result 402-1 in intermediate sample image 401-1 changes from the target object region TZ to the target object A displayed in the target object region TZ. That is, the target detection in intermediate sample image 401-1 is incorrect, and both the intermediate sample image and the adversarial sample determined based on the intermediate sample image have an attack effect.

[0067] For example, the following embodiment can be used to determine the target intermediate sample image based on the target object detection result of the intermediate sample image: based on the confidence threshold and the category confidence of the target object, the intermediate sample image with the category confidence of the target object being less than or equal to the confidence threshold is determined as the target intermediate sample image.

[0068] For each intermediate sample image, target detection can be performed to obtain the corresponding target object detection result. Based on a confidence threshold and the target object's category confidence, intermediate sample images with a category confidence of the target object less than or equal to the confidence threshold are identified as target intermediate sample images. This allows for the selection of the intermediate sample image with better attack performance from at least one intermediate sample image. The confidence threshold serves as the criterion for selecting target intermediate sample images. Subsequently, target adversarial sample images are generated based on these target intermediate sample images. These adversarial sample images are used, for example, to attack the target detection model. Therefore, to select target adversarial sample images with better attack performance, a lower confidence threshold can be set, for example.

[0069] For example, the category confidence of the target object in at least one intermediate sample image can be sorted, and a predetermined number of intermediate sample images with the lowest confidence can be determined as target intermediate sample images.

[0070] For example, the confidence threshold can be set to 0.25, or 25%.

[0071] The target object region displays the target object. Taking a vehicle as an example, for the target object detection task of identifying the object category within the target object region, some locations, such as the edges of the vehicle, better reflect its characteristics, and these locations contribute more to the identification of the object category. Conversely, some locations, such as the inside of the vehicle's cabin, contribute less to the identification of the object category. When the window mask is located in a region that contributes more to the identification of the object category, the probability of incorrect object category identification is higher, and the attack effect on the corresponding perturbation region and intermediate sample image is better. This allows for the identification of the target intermediate sample image.

[0072] In operation S423, the area where the sliding window corresponding to the intermediate sample image of the target is located is determined as the perturbation area.

[0073] exist Figure 4 In the example, target intermediate sample image 403-1 and target intermediate sample image 403-2 are schematically shown. It is also schematically shown that the area where the sliding window w1 corresponding to target intermediate sample image 403-1 is located is the perturbation area da-1, and the area where the sliding window wm corresponding to target intermediate sample image 403-2 is located is the perturbation area da-2.

[0074] like Figure 3 As shown, a specific example of generating a target adversarial example image from an intermediate sample image having a perturbation region based on at least one intermediate sample image is also schematically illustrated in operation S330. Figure 3 In the examples, target adversarial sample image 304-1 and target adversarial sample image 304-2 are schematically shown.

[0075] Figure 5 The illustration shows a specific example of how operation S530 generates a target adversarial sample image based on an intermediate sample image with a perturbation region in at least one intermediate sample image.

[0076] like Figure 5 As shown, for example, the following embodiment can be used to implement operation S530, which generates a target adversarial sample image based on an intermediate sample image with a perturbation region in at least one intermediate sample image.

[0077] In operation S531, an initial adversarial sample image is determined based on an intermediate sample image with a perturbation region.

[0078] For example, an intermediate sample image with perturbation regions can be determined as the initial adversarial example image. Figure 5In the example, it is illustrated that an intermediate sample image 501-1 with a perturbation region da-1 is identified as an initial adversarial sample image 502-1, and it is also illustrated that an intermediate sample image 501-2 with a perturbation region da-2 is identified as an initial adversarial sample image 502-2.

[0079] For example, the pixel values ​​of the perturbation region in the initial adversarial example image can be the initial pixel values. Figure 5 In the example, the initial pixel value pi-1 of the perturbation region da-1 of the initial adversarial sample image 502-1 is schematically shown, and the initial pixel value pi-2 of the perturbation region da-2 of the initial adversarial sample image 502-2 is also schematically shown.

[0080] In operation S532, the pixel values ​​of the perturbation region are adjusted according to the value of the sample generation loss function of the initial adversarial sample image. When the sample generation loss function converges, the target adversarial sample image is generated based on the pixel values ​​of the initial adversarial sample image and the perturbation region.

[0081] exist Figure 5 The example illustrates the generation of target adversarial sample images 503-1 and 503-2 when the sample generation loss function converges. The pixel value of the perturbation region da-1 in target adversarial sample image 503-1 is pt-1, and the pixel value of the perturbation region da-2 in target adversarial sample image 503-2 is pt-2.

[0082] The sample generation loss function Loss-S is related to the target object detection results of the initial adversarial sample image.

[0083] For example, the sample generation loss function Loss-S can be characterized using the following formula.

[0084]

[0085] in, The target object detector is used for target detection, and t represents the ground truth label of the current target.

[0086] Besides the location of the perturbation region, the pixel values ​​of the perturbation region also affect target object detection. According to the adversarial example generation method of this disclosure, when the sample loss function is correlated with the target object detection result of the initial adversarial example image, the pixel values ​​of the perturbation region are adjusted based on the value of the sample loss function. When the sample generation loss function converges, the pixel values ​​of the perturbation region can be learned to achieve a better attack effect, thus improving the attack effect of the target adversarial example image.

[0087] According to another embodiment of the adversarial sample generation method of the present disclosure, it may further include: performing target detection on each initial adversarial sample image to obtain the target object detection result of each initial adversarial sample image.

[0088] For example, the target object detection result of the initial adversarial example image includes the target object region and the target object category confidence. The target object region can be characterized, for example, using two parameters: length and width in pixels.

[0089] According to another embodiment of the adversarial example generation method of this disclosure, it may further include: performing image resizing preprocessing on the target adversarial example image. The image resizing preprocessing is set to gradient backpropagation.

[0090] For example, in scenarios involving target object detection using object detection models, the size of the adversarial sample image needs to be adjusted to accommodate the model's input parameters, such as sample image size. This resizing affects the entire adversarial sample image and consequently alters the size of the perturbation region. This can be achieved through methods like upsampling via interpolation or downsampling via averaging, thus adjusting the size of the adversarial sample image and the perturbation region. However, these upsampling and downsampling methods can alter the pixel values ​​and distribution of some pixels within the perturbation region, impacting the attack's effectiveness. Furthermore, in some cases, the upsampling and downsampling processes in the perturbation region are irreversible. The pixel values ​​and distribution of the perturbation region differ from those obtained after upsampling and downsampling, further complicating the image resizing preprocessing and impacting the attack effectiveness of the adversarial sample image.

[0091] According to the adversarial example generation method of this disclosure, the image size adjustment preprocessing of the target adversarial example image, and the image size adjustment preprocessing being set to gradient backpropagation, can be understood as follows: after the image adjustment preprocessing of the target adversarial example image, a target image can be obtained. Target detection is then performed on the target image to obtain the target object detection result of the target image. The value of the image size adjustment loss function can also be determined based on the target object detection result of the target image, the target object detection result of the target adversarial example image, and the image size adjustment loss function (the image size adjustment loss function is related to the target object detection result of the target image and the target object detection result of the target adversarial example image). Furthermore, the gradient can be determined based on the value of the image size adjustment loss function, and gradient backpropagation is performed to make the target object detection result of the target image tend to the target object detection result of the target adversarial example image.

[0092] The adversarial example generation method of this disclosure, by performing image resizing preprocessing on the target adversarial example image, can adapt to scenarios with specific requirements for the size of the target adversarial example image. By setting the image resizing preprocessing as gradient backpropagation, the process of resizing preprocessing of the target adversarial example image can be learned, more accurately restoring the pixel values ​​and pixel distribution of the target adversarial example image before image resizing preprocessing, and avoiding the weakening of the attack effect of the target adversarial example image due to image resizing preprocessing.

[0093] For example, the `interplote` function in PyTorch can be used instead of OpenCV's `resize` function to perform image resizing preprocessing on the target adversarial example image. The `interplote` function can perform image resizing preprocessing with gradient backpropagation.

[0094] For example, preprocessing such as channel replacement and pixel value normalization can also be performed on the target adversarial sample image.

[0095] Figure 6 A flowchart illustrating a training method for an object detection model according to an embodiment of the present disclosure is provided.

[0096] like Figure 6 As shown, a training method 600 for a target detection model according to an embodiment of the present disclosure includes operations S610 to S620.

[0097] In operation S610, multiple target detection sample images are input into the initial target detection model to obtain the target object detection results.

[0098] The multiple target detection sample images include at least one target adversarial sample image, which is obtained using the adversarial sample image generation method according to the above embodiments.

[0099] For example, the multiple target detection sample images may also include at least one initial sample image.

[0100] For example, the initial object detection model may include, for instance, the YOLO (You Only Look Once) model or the Fast-RCNN (Fast-Region based CNN) model.

[0101] When operating the S620, the model parameters of the initial object detection model are adjusted according to the value of the model training loss function to obtain the object detection model.

[0102] The model training loss function is related to the target object detection results and labels of the target detection sample images. The value of the model training loss function characterizes the accuracy and performance of the initial target detection model for the sample images. By adjusting the model parameters of the initial target detection model based on the value of the model training loss function, a target detection model with higher accuracy and better performance can be obtained. The trained target detection model can be applied to fields such as autonomous driving, improving the safety and robustness of the target detection model in the real world.

[0103] Figure 7 A flowchart of a target detection method according to an embodiment of the present disclosure is shown as an example.

[0104] like Figure 7 As shown, a target detection method 700 according to an embodiment of the present disclosure includes operation S710.

[0105] When operating the S710, the image to be detected is input into the target detection model to obtain the target object detection result.

[0106] The object detection model is obtained using the training method described above.

[0107] According to the target detection method of this disclosure, since the target detection model is obtained by using the training method of the target detection model described above, the target detection result obtained by inputting the image to be detected into the target detection model is more accurate and has higher robustness.

[0108] This disclosure also proposes an adversarial sample image generation apparatus.

[0109] Figure 8 A block diagram of an adversarial sample image generation apparatus according to an embodiment of the present disclosure is shown as an example.

[0110] like Figure 8 As shown, the adversarial sample image generation apparatus 800 according to an embodiment of the present disclosure includes: an intermediate sample image determination module 810, a disturbance region determination module 820, and a target adversarial sample image generation module 830.

[0111] The intermediate sample image determination module 810 is used to perform sliding windowing on the target object area of ​​the initial sample image using a window mask to obtain at least one intermediate sample image.

[0112] The perturbation region determination module 820 is used to determine at least one perturbation region based on the target object detection result for at least one intermediate sample image and the corresponding sliding window.

[0113] The target adversarial sample image generation module 830 is used to generate a target adversarial sample image based on an intermediate sample image with a perturbation region in at least one intermediate sample image.

[0114] According to embodiments of this disclosure, the target adversarial sample image generation module includes: an initial adversarial sample image determination submodule and a target adversarial sample image generation submodule.

[0115] The initial adversarial sample image determination submodule is used to determine the initial adversarial sample image based on the intermediate sample image with perturbation regions.

[0116] The pixel values ​​of the perturbation region in the initial adversarial sample image are the initial pixel values.

[0117] The target adversarial sample image generation submodule is used to adjust the pixel values ​​of the perturbation region based on the value of the sample generation loss function of the initial adversarial sample image, and generate the target adversarial sample image based on the pixel values ​​of the initial adversarial sample image and the perturbation region when the sample generation loss function converges.

[0118] The sample generation loss function is related to the target object detection results of the initial adversarial sample image.

[0119] According to embodiments of this disclosure, the disturbance region determination module includes: a first target detection submodule, a target intermediate sample image determination submodule, and a disturbance region determination submodule.

[0120] The first object detection submodule is used to perform object detection on each intermediate sample image and obtain the object detection result of each intermediate sample image.

[0121] The target intermediate sample image determination submodule is used to determine the target intermediate sample image based on the target object detection results of the intermediate sample image.

[0122] The perturbation region determination submodule is used to determine the area where the sliding window corresponding to the intermediate sample image of the target is located as the perturbation region.

[0123] According to embodiments of this disclosure, the target object detection result includes the category confidence of the target object; the target intermediate sample image determination submodule includes: a target intermediate sample image determination unit, used to determine intermediate sample images whose category confidence of the target object is less than or equal to the confidence threshold as target intermediate sample images based on the confidence threshold and the category confidence of the target object.

[0124] The adversarial sample image generation apparatus according to embodiments of the present disclosure further includes: a second target detection module, used to perform target detection on each initial adversarial sample image to obtain the target object detection result of each initial adversarial sample image.

[0125] The target object detection results of the initial adversarial sample image include the target object region and the target object category confidence.

[0126] The adversarial sample image generation apparatus according to an embodiment of the present disclosure further includes: an image size adjustment preprocessing module, used to perform image size adjustment preprocessing on the target adversarial sample image, wherein the image size adjustment preprocessing is configured as gradient backpropagation.

[0127] This disclosure also proposes a training apparatus for an object detection model.

[0128] Figure 9 A block diagram of an object evaluation apparatus according to an embodiment of the present disclosure is shown as an example.

[0129] like Figure 9 As shown, the training apparatus 900 for the target detection model according to an embodiment of the present disclosure includes: a target object detection result determination module 910 and a model parameter adjustment module 920.

[0130] The target object detection result determination module is used to input multiple target detection sample images into the initial target detection model to obtain the target object detection result.

[0131] The plurality of target detection sample images include at least one target adversarial sample image, which is obtained using an adversarial sample image generation apparatus according to an embodiment of the present disclosure.

[0132] The model parameter adjustment module is used to adjust the model parameters of the initial object detection model based on the value of the model training loss function, so as to obtain the object detection model.

[0133] The model training loss function is related to the target object detection results and labels of the target detection sample images.

[0134] This disclosure also proposes a training device for a target detection apparatus.

[0135] Figure 10 A block diagram of a target detection apparatus according to an embodiment of the present disclosure is shown as an example.

[0136] like Figure 10 As shown, the target detection device 1000 according to the target detection model of the present disclosure includes: a target detection module 1010, used to input the image to be detected into the target detection model to obtain the target object detection result.

[0137] The object detection model is obtained using a training apparatus for the object detection model according to embodiments of the present disclosure.

[0138] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0139] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0140] like Figure 11 As shown, device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1102 or a computer program loaded into random access memory (RAM) 1103 from storage unit 1108. The RAM 1103 may also store various programs and data required for the operation of device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0141] Multiple components in device 1100 are connected to I / O interface 1105, including: input unit 1106, such as keyboard, mouse, etc.; output unit 1107, such as various types of monitors, speakers, etc.; storage unit 1108, such as disk, optical disk, etc.; and communication unit 1109, such as network card, modem, wireless transceiver, etc. Communication unit 1109 allows device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0142] The computing unit 1101 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as data sample generation methods, object evaluation methods, and model training methods. For example, in some embodiments, the data sample generation methods, object evaluation methods, and model training methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of the data sample generation methods, object evaluation methods, and model training methods described above can be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured in any other suitable manner (e.g., by means of firmware) to perform an adversarial example image generation method, a target detection model training method, and a target detection method.

[0143] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0145] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0146] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0147] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0148] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0149] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0150] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating adversarial example images, comprising: Using a window mask, a sliding window is applied to the target object region of the initial sample image to obtain at least one intermediate sample image; Based on the target object detection results for the at least one intermediate sample image and the corresponding sliding window, at least one perturbation region is determined; The disturbance region is the area where the window mask affects the detection result of the target object; The disturbance region has an attack effect on target object detection; and Based on the intermediate sample image containing the perturbation region in the at least one intermediate sample image, a target adversarial sample image is generated; The step of generating a target adversarial sample image from the intermediate sample image containing the perturbation region in the at least one intermediate sample image includes: An initial adversarial sample image is determined based on an intermediate sample image having the perturbation region, wherein the pixel values ​​of the perturbation region in the initial adversarial sample image are initial pixel values; and Based on the value of the sample generation loss function of the initial adversarial sample image, the pixel values ​​of the perturbation region are adjusted. If the sample generation loss function converges, a target adversarial sample image is generated based on the initial adversarial sample image and the pixel values ​​of the perturbation region. The sample generation loss function is related to the target object detection result of the initial adversarial sample image and is related to the attack effect on the target object detection. The target adversarial sample image is preprocessed by image resizing to obtain the target image. The image resizing preprocessing is set to gradient backpropagation. The gradient backpropagation includes: determining the value of the image resizing loss function based on the target object detection result of the target image, the target object detection result of the target adversarial sample image, and the image resizing loss function; determining the gradient based on the value of the image resizing loss function; and performing gradient backpropagation so that the target object detection result of the target image tends to the target object detection result of the target adversarial sample image.

2. The method according to claim 1, wherein, Determining at least one perturbation region based on the target object detection results and the corresponding sliding window for the at least one intermediate sample image includes: Perform target detection on each of the intermediate sample images to obtain the target object detection result for each of the intermediate sample images; Based on the target object detection results of the intermediate sample image, the target intermediate sample image is determined; and The area where the sliding window corresponding to the target intermediate sample image is located is determined as the perturbation region.

3. The method according to claim 2, wherein, The target object detection result includes the category confidence score of the target object; determining the target intermediate sample image based on the target object detection result of the intermediate sample image includes: Based on the confidence threshold and the category confidence of the target object, the intermediate sample images whose category confidence of the target object is less than or equal to the confidence threshold are determined as the target intermediate sample images.

4. The method according to claim 1, further comprising: Target detection is performed on each of the initial adversarial sample images to obtain the target object detection result of each initial adversarial sample image, wherein the target object detection result of the initial adversarial sample image includes the target object region and the category confidence of the target object.

5. A method for training an object detection model, comprising: Multiple target detection sample images are input into an initial target detection model to obtain target object detection results. The multiple target detection sample images include at least one adversarial sample image, which is obtained using the adversarial sample image generation method according to any one of claims 1-4. Based on the value of the model training loss function, the model parameters of the initial object detection model are adjusted to obtain the object detection model, wherein the model training loss function is related to the target object detection result and label of the object detection sample image.

6. A target detection method, comprising: The image to be detected is input into the target detection model to obtain the target object detection result. The target detection model is obtained using the training method for the target detection model according to claim 5.

7. An adversarial example image generation apparatus, comprising: The intermediate sample image determination module is used to perform sliding windowing in the target object region of the initial sample image using a window mask to obtain at least one intermediate sample image. The perturbation region determination module is used to determine at least one perturbation region based on the target object detection result and the corresponding sliding window for the at least one intermediate sample image; The disturbance region is the area where the window mask affects the detection result of the target object; The disturbance region has an attack effect on target object detection; and The target adversarial sample image generation module is used to generate a target adversarial sample image based on the intermediate sample image containing the perturbation region in the at least one intermediate sample image; An image resizing preprocessing module is used to perform image resizing preprocessing on the target adversarial sample image to obtain a target image. The image resizing preprocessing is set to gradient backpropagation. The gradient backpropagation includes: determining the value of the image resizing loss function based on the target object detection result of the target image, the target object detection result of the target adversarial sample image, and the image resizing loss function; determining the gradient based on the value of the image resizing loss function; and performing gradient backpropagation so that the target object detection result of the target image tends to the target object detection result of the target adversarial sample image. The target adversarial example image generation module includes: An initial adversarial example image determination submodule is configured to determine an initial adversarial example image based on an intermediate sample image having the perturbation region, wherein the pixel values ​​of the perturbation region in the initial adversarial example image are initial pixel values; and The target adversarial sample image generation submodule is used to adjust the pixel values ​​of the perturbation region according to the value of the sample generation loss function of the initial adversarial sample image, and generate a target adversarial sample image based on the initial adversarial sample image and the pixel values ​​of the perturbation region when the sample generation loss function converges. The sample generation loss function is related to the target object detection result of the initial adversarial sample image and is related to the attack effect on the target object detection.

8. The apparatus according to claim 7, wherein, The disturbance region determination module includes: The first target detection submodule is used to perform target detection on each of the intermediate sample images to obtain the target object detection result of each of the intermediate sample images; The target intermediate sample image determination submodule is used to determine the target intermediate sample image based on the target object detection results of the intermediate sample image; and The perturbation region determination submodule is used to determine the area where the sliding window corresponding to the target intermediate sample image is located as the perturbation region.

9. The apparatus according to claim 8, wherein, The target object detection result includes the target object's category confidence score; the target intermediate sample image determination submodule includes: The target intermediate sample image determination unit is used to determine the intermediate sample images whose category confidence of the target object is less than or equal to the confidence threshold as the target intermediate sample images based on the confidence threshold and the category confidence of the target object.

10. The apparatus according to claim 7, further comprising: The second target detection module is used to perform target detection on each of the initial adversarial sample images to obtain the target object detection result of each of the initial adversarial sample images, wherein the target object detection result of the initial adversarial sample image includes the target object region and the category confidence of the target object.

11. A training device for an object detection model, comprising: A target object detection result determination module is used to input multiple target detection sample images into an initial target detection model to obtain target object detection results, wherein the multiple target detection sample images include at least one target adversarial sample image, and the target adversarial sample image is obtained using the adversarial sample image generation device according to any one of claims 7-10; and The model parameter adjustment module is used to adjust the model parameters of the initial object detection model according to the value of the model training loss function to obtain the object detection model, wherein the model training loss function is related to the target object detection result and label of the object detection sample image.

12. A target detection device, comprising: The object detection module is used to input the image to be detected into the object detection model and obtain the object detection results. The target detection model is obtained using the training device for the target detection model according to claim 11.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

15. A computer program product comprising a computer program stored on at least one of a readable storage medium and an electronic device, the computer program implementing the method according to any one of claims 1-6 when executed by a processor.

Images