Adversarial image generation method, device, electronic device and storage medium

Abstract

The present disclosure provides an adversarial image generation method, device, electronic device and storage medium, which relate to the fields of computer vision technology, in particular to deep learning and autonomous driving technology. The specific implementation scheme is as follows: using a first patch image to perform occlusion processing on a target sample image in a sample image set, a plurality of first images with different disturbance area positions are obtained; then target detection is performed on the plurality of first images to obtain M second images and a plurality of candidate position information of the disturbance area; for the mth second image, the size of the first patch image is adjusted to obtain a plurality of third images with different disturbance area sizes; then target detection is performed on the plurality of third images to determine a plurality of candidate sizes of the disturbance area; based on the plurality of candidate position information and the plurality of candidate sizes, disturbance information is added to the sample image to obtain a candidate adversarial image set; target detection is performed on the candidate adversarial image set and the sample image set to obtain a target adversarial image.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer vision technology, in particular to deep learning and autonomous driving technology, and specifically to a method, device, electronic device, and storage medium for generating an adversarial image. Background Art

[0002] Adversarial images are images that contain intentionally or deliberately perturbed pixels to confuse or deceive the model. Adversarial images can cause deep neural networks to make incorrect predictions.

[0003] With the widespread application of target detection technology in the field of autonomous driving safety, the security requirements for target detection models are becoming increasingly higher. Summary of the Invention

[0004] The present disclosure provides a method, device, electronic device and storage medium for generating an adversarial image.

[0005] According to one aspect of the present disclosure, a method for generating an adversarial image is provided, comprising: performing occlusion processing on a target sample image in a sample image set using a first patch image to obtain a plurality of first images with different positions of disturbance regions; and performing target detection on the plurality of first images to obtain M second images and a plurality of candidate position information of disturbance regions corresponding to the M second images, wherein M is an integer greater than or equal to 1. For the mth second image, adjusting the size of the first patch image to obtain a plurality of third images with different sizes of disturbance regions; and performing target detection on the plurality of third images to determine a plurality of candidate sizes of disturbance regions, wherein m is an integer greater than or equal to 1 and less than or equal to M. Based on the plurality of candidate position information and the plurality of candidate sizes, disturbance information is added to the sample images in the sample image set to obtain a candidate adversarial image set. And by performing target detection on the candidate adversarial image set and the sample image set, a target adversarial image is obtained from the candidate adversarial image set.

[0006] According to another aspect of the present disclosure, a device for generating an adversarial image is provided, comprising: a first processing module, a second processing module, a first obtaining module, and a second obtaining module. The first processing module is configured to perform occlusion processing on a target sample image in a sample image set using a first patch image to obtain multiple first images with different perturbation region positions; and to obtain M second images and multiple candidate position information of perturbation regions corresponding to the M second images by performing target detection on the multiple first images, where M is an integer greater than or equal to 1. The second processing module is configured to adjust the size of the first patch image for the mth second image to obtain multiple third images with different perturbation region sizes; and to determine multiple candidate sizes of perturbation regions by performing target detection on the multiple third images, where m is an integer greater than or equal to 1 and less than or equal to M. The first obtaining module is configured to add perturbation information to sample images in the sample image set based on the multiple candidate position information and the multiple candidate sizes to obtain a candidate adversarial image set. The second obtaining module is configured to obtain a target adversarial image from the candidate adversarial image set by performing target detection on the candidate adversarial image set and the sample image set.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described above.

[0009] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method described above when executed by a processor.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0012] Figure 1 Schematically illustrates an exemplary system architecture to which the adversarial image generation method and apparatus according to an embodiment of the present disclosure may be applied;

[0013] Figure 2The flowchart of the adversarial image generation method according to an embodiment of the present disclosure is schematically shown;

[0014] Figure 3 A schematic diagram schematically illustrates how to perform occlusion processing on a target sample image using a first patch image to obtain a first image according to an embodiment of the present disclosure;

[0015] Figure 4 Schematically illustrating a schematic diagram of adjusting the size of a first patch image to obtain a third image according to an embodiment of the present disclosure;

[0016] Figure 5 A schematic diagram of obtaining a candidate adversarial image set according to an embodiment of the present disclosure is schematically shown;

[0017] Figure 6 A schematic diagram of obtaining a target adversarial image according to an embodiment of the present disclosure is schematically shown;

[0018] Figure 7 A block diagram schematically illustrates a method for generating an adversarial image according to an embodiment of the present disclosure; and

[0019] Figure 8 A block diagram of an electronic device suitable for implementing the adversarial image generation method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0020] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] To test the effectiveness of object detection models in autonomous driving safety, we can add interference factors to the target image to generate adversarial images. These patches are then added to real-world images for image attack testing. If the object detection model misdetects the target object or its category in the target image, the image attack is considered successful.

[0022] Image attack testing can verify the security of the target detection model algorithm and improve the robustness of the target detection model.

[0023] In related technologies, when adding perturbation information to a target image, the size of the patch image and the relative position of the patch image within the target image are generally randomly specified based on human experience. However, this approach has certain limitations. When the target to be detected is a relatively complex and large object, such as a large vehicle, the size and position of the patch image have a significant impact on the success rate of the image attack, considering the local correlation of the target detection model with the internal area of the target. Using only the patch image randomly specified by human experience to add perturbation information to the target image, the adversarial image generated has a low success rate in the image attack.

[0024] In view of this, an embodiment of the present disclosure provides an adversarial image generation method, comprising: using a first patch image to perform occlusion processing on a target sample image in a sample image set to obtain a plurality of first images with different disturbance region positions; and performing target detection on the plurality of first images to obtain M second images and a plurality of candidate position information of disturbance regions corresponding to the M second images, wherein M is an integer greater than or equal to 1. For the mth second image, adjusting the size of the first patch image to obtain a plurality of third images with different disturbance region sizes; and performing target detection on the plurality of third images to determine a plurality of candidate sizes of the disturbance region, wherein m is an integer greater than or equal to 1 and less than or equal to M. Based on the plurality of candidate position information and the plurality of candidate sizes, disturbance information is added to the sample images in the sample image set to obtain a candidate adversarial image set. By performing target detection on the candidate adversarial image set and the sample image set, a target adversarial image is obtained from the candidate adversarial image set.

[0025] Figure 1 An exemplary system architecture to which the adversarial image generation method and apparatus according to an embodiment of the present disclosure can be applied is schematically shown.

[0026] It should be noted that Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure. This does not mean that the embodiments of the present disclosure cannot be applied to other devices, systems, environments, or scenarios. For example, in another embodiment, the exemplary system architecture to which the adversarial image generation method and apparatus can be applied may include a terminal device, but the terminal device can implement the adversarial image generation method and apparatus provided by the embodiments of the present disclosure without interacting with a server.

[0027] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0029] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0030] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports the content browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0031] It should be noted that the adversarial image generation method provided in the embodiments of the present disclosure can generally be executed by the terminal device 101, 102, or 103. Accordingly, the adversarial image generation apparatus provided in the embodiments of the present disclosure can also be provided in the terminal device 101, 102, or 103.

[0032] Alternatively, the adversarial image generation method provided in the embodiments of the present disclosure may also be generally executed by the server 105. Accordingly, the adversarial image generation apparatus provided in the embodiments of the present disclosure may generally be provided in the server 105. The adversarial image generation method provided in the embodiments of the present disclosure may also be performed by a server or server cluster that is different from the server 105 and that is capable of communicating with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the adversarial image generation apparatus provided in the embodiments of the present disclosure may also be provided in a server or server cluster that is different from the server 105 and that is capable of communicating with the terminal devices 101, 102, 103 and / or the server 105.

[0033] For example, the terminal devices 101, 102, and 103 can obtain sample images of different scales for different scenes, and then send the obtained target content to the server 105, which analyzes the sample images, for example: classifying the sample images by scene, and determining the target sample image from the classified sample images in the same scene; using the first patch image to perform occlusion processing on the target sample image to obtain multiple first images with different positions of the disturbance area, and by performing target detection on the multiple first images, obtain multiple second images and multiple candidate position information of the disturbance area; then, for each second image, adjust the size of the first patch image to obtain multiple third images with different sizes of the disturbance area; and by performing target detection on the multiple third images, determine multiple candidate sizes of the disturbance area; then, based on the multiple candidate position information and the multiple candidate sizes, add disturbance information to the sample images in the sample image set to obtain a candidate adversarial image set; by performing target detection on the candidate adversarial image set and the sample image set, obtain the target adversarial image from the candidate adversarial image set. Alternatively, a server or server cluster that can communicate with the terminal devices 101 , 102 , 103 and / or the server 105 processes the target sample image and finally obtains a target adversarial image.

[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0035] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information involved comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0036] In the technical solution disclosed herein, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0037] Figure 2 The flowchart of the adversarial image generation method according to an embodiment of the present disclosure is schematically shown.

[0038] like Figure 2 As shown, the method includes operations S210 to S240.

[0039] In operation S210, a target sample image in the sample image set is occluded using a first patch image to obtain a plurality of first images having different disturbance region positions. Target detection is performed on the plurality of first images to obtain M second images and a plurality of candidate position information of disturbance regions corresponding to the M second images, where M is an integer greater than or equal to 1.

[0040] In operation S220, for the mth second image, the size of the first patch image is adjusted to obtain a plurality of third images having different sizes of disturbed regions; and by performing target detection on the plurality of third images, a plurality of candidate sizes of the disturbed regions are determined, where m is an integer greater than or equal to 1 and less than or equal to M.

[0041] In operation S230 , disturbance information is added to the sample images in the sample image set based on the plurality of candidate position information and the plurality of candidate sizes to obtain a candidate adversarial image set.

[0042] In operation S240 , target adversarial images are obtained from the candidate adversarial image set by performing object detection on the candidate adversarial image set and the sample image set.

[0043] According to embodiments of the present disclosure, a sample image set may include sample images of different scales and scenes. The scale may represent the image acquisition distance between the image acquisition device and the target object. The sample image set may be divided according to different scenes, resulting in multiple sample images of different scales within the same scene. The sample image with the highest resolution among the multiple sample images is then determined as the target sample image.

[0044] According to an embodiment of the present disclosure, the first patch image can be a pure white or pure black image. The initial size of the first patch image can be determined based on the target detection frame size of the target object in the target sample image. The initial size may include an initial length and an initial width. For example, the initial length of the first patch image can be 1 / 3 of the length of the target detection frame, and the initial width of the first patch image can be 1 / 3 of the width of the target detection frame.

[0045] According to an embodiment of the present disclosure, configuration information may be set according to empirical values of patch sizes for different target objects, and an initial size of the patch image may be obtained from the configuration information according to the category of the target object in the target sample image.

[0046] According to embodiments of the present disclosure, a target sample image can be occluded using a first patch image. By varying the relative position of the first patch image and the target sample image, multiple first images with different disturbed regions can be obtained. The disturbed region represents the occluded region of the target sample image by the first patch image.

[0047] For example, the first patch image can be a rectangular pure black image, and the pixel coordinates of the upper left corner of the rectangle can be used to represent the position of the disturbed area. Each time the relative position between the first patch image and the target sample image is transformed, a first image and the position coordinates (x1, y1) of the disturbed area in the first image can be obtained. The same logic can be applied, and no further explanation is given here.

[0048] According to an embodiment of the present disclosure, target detection is performed on a first image to obtain a target detection result. The target detection result can represent the confidence level that the target object is correctly identified. M second images can be obtained from multiple first images by setting a confidence threshold range.

[0049] For example, the confidence threshold range may be 0.25, and the first image may include images P1, P2, and P3, where the disturbed regions have different positions. The coordinates of the disturbed region position of image P1 may be (x p1 ,y p1 ), the coordinates of the disturbed area of image P2 can be (x p2 ,y p2 ), the coordinates of the disturbed area of image P3 can be (x p3 ,y p3 ). By performing target detection on images P1, P2, and P3, the confidence levels of the target objects being correctly identified are 0.4, 0.2, and 0.15, respectively. Images P2 and P3 can be determined as the second images, and the candidate position information of the disturbance area can be: (x p2 ,y p2 )、(x p3 ,y p3 ).

[0050] According to an embodiment of the present disclosure, for the mth second image, in the mth second image, the position coordinates of the disturbance area may be (x pm ,y pm ), the initial size of the first patch image may be (w0, h0), and the size of the first patch image may be adjusted to obtain a plurality of third images with different disturbance region sizes, for example: image P m1 The size of the disturbance region can be (w pm1 , h pm1 ), image P m2 The size of the disturbance region can be (w pm2 , h pm2 ), image P m3 The size of the disturbance region can be (w pm3 , h pm3 ). By m1 , image P m2 and image P m3 The target detection is performed separately, and the confidence levels of the target objects being correctly identified are 0.15, 0.3, and 0.2 respectively. m2 and image P m3 Determined as the third image, the candidate size information of the disturbance area can be (w pm1 , h pm1 )、(wpm3 , h pm3 ).

[0051] According to an embodiment of the present disclosure, the disturbance information may include size information of the disturbance region and position information of the disturbance region. The disturbance information may be obtained by randomly combining different candidate position information and different candidate size information.

[0052] For example, based on the candidate location information of the mth disturbance area (x pm ,y pm ) and candidate size information of the disturbance region (w pm1 , h pm1 ), add perturbation information to the sample image and obtain the mth candidate adversarial image. Similarly, a set of candidate adversarial images can be obtained.

[0053] According to an embodiment of the present disclosure, a target adversarial image can be obtained from a set of candidate adversarial images by setting a threshold range of confidence.

[0054] For example, the confidence threshold range can be 0.25. When the candidate adversarial image causes the target object's confidence in its true value to be less than 0.25, the target attack is considered successful. At this point, the target detection results show that the target object has disappeared or has been misclassified as another category by the target detection model. This candidate adversarial image can be determined as the target adversarial image.

[0055] According to an embodiment of the present disclosure, target detection is first performed on a plurality of first images with different positions of the disturbance region to obtain a plurality of candidate position information of the disturbance region and a second image corresponding to each candidate position information. Then, by adjusting the size of the first patch image in the second image, a plurality of third images with different sizes of the disturbance region are obtained, and target detection is performed to obtain a plurality of candidate sizes of the disturbance region. For the candidate adversarial image set obtained based on the candidate position and candidate size of the disturbance region, target detection is performed on the candidate adversarial image set and the sample image set to filter the target adversarial image from the candidate adversarial image set. Since the candidate position information of the disturbance region and the candidate size information of the disturbance region are both obtained based on the results of target detection, the position and size of the disturbance region are adaptively selected so that the position and size of the disturbance region in the final target adversarial image have a relatively high impact on the performance of target detection, thereby effectively improving the success rate of the adversarial image attack on the target detection model.

[0056] Reference below Figures 3 to 6 , combined with specific embodiments Figure 2 The method shown is further explained.

[0057] According to an embodiment of the present disclosure, the operation S210 may include the following operations:

[0058] According to the position information of the target object in the target sample image, first starting position information of the first patch image is determined. According to the first starting position information, a sliding window operation is performed on the target sample image using the first patch image with a first step length to obtain multiple first images.

[0059] According to an embodiment of the present disclosure, the position information of the target object can represent the position information of the target detection frame obtained by performing target detection on the target sample image. The first starting position of the first patch image can be the upper left corner of the target detection frame, and the first starting position information can be the pixel coordinate information of the upper left corner of the target detection frame. Using the first patch image, a sliding window operation is performed on the target sample image with a first step length to obtain the first image corresponding to each sliding window position.

[0060] According to an embodiment of the present disclosure, based on the first starting position information, a sliding window operation is performed on the target sample image using the first patch image with the first step length to obtain multiple first images. The following operations may be performed:

[0061] Determine the first step length and the first moving direction. Utilize the first patch image and perform a sliding window operation on the target sample image along the first moving direction with the first step length to obtain a plurality of first images.

[0062] According to an embodiment of the present disclosure, the information of the first moving direction may represent the direction in which the first patch image moves relative to the target sample image when performing a sliding window operation.

[0063] For example, the pixel coordinates of the upper left corner of the target detection frame in the target sample image can be (x0, y0), the first step length can be 2 pixels, and the first movement direction can be the lower right. Then, a sliding window operation is performed on the target sample image using the first patch image. In the resulting first image, the pixel coordinates of the upper left corner of the first patch image can be (x0+2, y0-2).

[0064] According to an embodiment of the present disclosure, the operation S210 may include the following operations:

[0065] Perform object detection on the multiple first images to obtain a first detection result, where the first detection result represents the confidence level that the target object is correctly identified. Based on the first detection result, M second images are obtained from the multiple first images. For the mth second image, determine the mth candidate position information of the disturbance region based on the position information of the first patch image in the mth second image.

[0066] Figure 3 A schematic diagram schematically illustrates how to obtain a first image by performing occlusion processing on a target sample image using a first patch image according to an embodiment of the present disclosure.

[0067] like Figure 3 As shown, in embodiment 300, a plurality of first images 314 are obtained by performing occlusion processing on a target object 312 in a target sample image 311 using a first patch image 313. The plurality of first images 314 may include T images, namely, image 314_1, image 314_2, ..., image 314_t, ..., image 314_T. In image 314_1, the first patch image 313 is located at the upper left corner of the target object 312. In image 314_2, the first patch image 313 is located at the left middle of the target object 312. In image 314_t, the first patch image 313 is located at the lower middle of the target object 312. In image 314_T, the first patch image 313 is located at the lower right corner of the target object 312. The relative position between the first patch image 313 and the target object 312 is the position of the disturbance area. It can be seen that the images 314_1, 314_2, ..., 314_t, ..., and 314_T have different disturbance areas.

[0068] According to an embodiment of the present disclosure, by performing object detection on image 314_1, image 314_2, ..., image 314_t, ..., and image 314_T, a confidence level that the target object in each image is correctly identified can be obtained. Based on the confidence level, M second images 315 are selected from the plurality of first images 314.

[0069] For example, by performing target detection on image 314_1, the confidence level of the target object being correctly identified may be 0.2. Taking the confidence level screening condition as less than 0.25 as an example, image 314_1 is determined to be a second image 315_1.

[0070] According to the embodiments of the present disclosure, a sliding window operation is performed on the target detection image using a fixed-size first patch image to obtain multiple first images with different perturbed regions. Target detection is then performed on these multiple first images to generate a second image. By preliminarily determining the degree to which the location of the perturbed region affects target detection performance, candidate location information for the perturbed region can be obtained, objectively providing perturbation information in the positional dimension for generating adversarial images.

[0071] During the target detection process, different regions of the target object contribute differently to the target detection results. Especially for images of different scales in the same scene, the size of the perturbed area is also an important factor affecting the effect of image attacks.

[0072] According to an embodiment of the present disclosure, the above operation S220 may include the following operations:

[0073] The second starting position information for resizing the first patch image is determined based on the position information of the first patch image in the mth second image. The first patch image is resized with a second step size based on the second starting position information to obtain a plurality of third images.

[0074] According to an embodiment of the present disclosure, the second starting position information may be pixel coordinate information of the lower right corner of the first patch image.

[0075] According to an embodiment of the present disclosure, starting from the lower right corner of the first patch image, the size of the disturbed area can be used as an independent variable, and the size of the disturbed area can be gradually increased and / or decreased with a second step size to obtain multiple third images.

[0076] According to an embodiment of the present disclosure, adjusting the size of the first patch image with a second step size according to the second starting position information to obtain a plurality of third images may include the following operations:

[0077] Determine information about a second step length and a second moving direction, and adjust the size of the first patch image along the second moving direction with a second step length to obtain a plurality of third images.

[0078] According to an embodiment of the present disclosure, the second moving direction may be an upper left direction.

[0079] For example, the size of the first patch image may include a length w0 and a width h0 of the first patch image. The second step size may be 5 pixels. After performing a sliding window operation, the length of the second patch image may be (w0+5) and the width may be (h0+5).

[0080] According to an embodiment of the present disclosure, performing object detection on multiple third images to determine multiple candidate sizes of the disturbance region may include the following operations:

[0081] Performing object detection on the plurality of third images to obtain a second detection result, wherein the second detection result represents a confidence level that the target object is correctly identified. Based on the second detection result, obtaining N fourth images from the plurality of third images, where N is an integer greater than or equal to 1. For the nth fourth image, determining an nth candidate size information of the disturbed region based on the size information of the second patch image in the nth fourth image, where n is an integer greater than or equal to 1 and less than or equal to N.

[0082] Figure 4 A schematic diagram schematically illustrates adjusting the size of a first patch image to obtain a third image according to an embodiment of the present disclosure.

[0083] like Figure 4As shown, in embodiment 400, for the mth second image 315_1, the first patch image 313 in the mth second image 315_1 is resized along the second moving direction with a second step size to obtain multiple third images 421. For example, the multiple third images may include P images, namely images 421_1, ..., and image 421_P. In image 421_1, the first patch image after resizing is patch image 422_1. In image 421_P, the first patch image after resizing is patch image 422_P.

[0084] According to an embodiment of the present disclosure, taking image 421_1 and image 421_P as an example, the position of the lower right corner of patch image 422_1 in image 421_1 is the same as the position of the lower right corner of patch image 422_P in image 421_P, that is, the second starting position for adjusting the size of the first patch image.

[0085] According to an embodiment of the present disclosure, by performing target detection on images 421_1, ..., and image 421_P, a confidence level that the target object in each image is correctly identified can be obtained. Based on the confidence level, N fourth images 423 are selected from the plurality of second images 421.

[0086] For example, by performing target detection on image 421_1, the confidence level of the target object being correctly identified may be 0.1. Taking the confidence level screening condition of less than 0.25 as an example, image 423_1 is determined to be a fourth image 423_1.

[0087] According to the embodiments of the present disclosure, after determining candidate locations for disturbance regions, the first patch image is resized for any second image with a fixed disturbance region, resulting in multiple third images with different disturbance regions. Target detection is then performed on these third images to determine candidate sizes for the disturbance regions. By further adjusting the sizes of the disturbance regions, more accurate information about the disturbance regions is obtained based on the target detection results, adapting to detection scenarios involving complex and large target objects and improving the success rate of image-based attacks.

[0088] Because the patch image is pure black or white, the black and white pixel values of the patch image and the pixel values of the perturbed region in the adversarial image may differ slightly in the target detection results. After filtering the perturbed region's location and size for the target sample image, the number of candidate locations and sizes of the perturbed region obtained is relatively small. Based on the candidate locations and sizes, perturbation information can be added to the sample image to obtain a set of candidate adversarial images.

[0089] According to an embodiment of the present disclosure, the plurality of candidate position information includes M pieces, and the plurality of candidate size information includes S pieces, where M and S are both integers greater than or equal to 1. Operation S230 may include the following operations:

[0090] Based on the mth candidate position information and the sth candidate size information, perturbation information of the target sample image is obtained, where s is an integer greater than or equal to 1 and less than or equal to S. Based on the perturbation information of the target sample image and the target detection box information of the first sample image, perturbation information of the first sample image is obtained, where the first sample image represents any sample image in the sample image set except the target sample image. The perturbation information of the target sample image is added to the target sample image, and the perturbation information of the first sample image is added to the first sample image, to obtain a candidate adversarial image set.

[0091] According to an embodiment of the present disclosure, the disturbance information represents the location information and size information of the disturbance area. For example, the mth candidate location information may be (x m ,y m ), the sth candidate size information can be (w s 、h s ). The perturbation information of the target sample image can be (x m ,y m , w s 、h s )

[0092] According to an embodiment of the present disclosure, obtaining disturbance information of the first sample image according to disturbance information of the target sample image and target detection frame information of the first sample image may include the following operations:

[0093] Obtain perturbation offset information based on the perturbation information of the target sample image and the target detection frame information of the target sample image. Obtain perturbation information of the first sample image based on the perturbation offset information and the target detection frame information of the first sample image.

[0094] According to an embodiment of the present disclosure, the target detection frame information of the target sample image may include the position coordinate information (x, y) of the upper left corner of the target detection frame and the size information of the target detection frame. The size information of the target detection frame may include the length information w and width information h of the target detection frame. Therefore, the target detection frame information can be expressed as (x, y, w, h).

[0095] According to an embodiment of the present disclosure, the disturbance offset information may include a position offset ratio and a size offset ratio of the disturbance area offset from the current target detection frame.

[0096] According to an embodiment of the present disclosure, the position offset ratio can be determined based on the position information of the disturbance area, the position information of the target detection frame, and the length information of the target detection frame, as shown in equations (1-1) and (1-2):

[0097] S x =(x m -x) / w; (1-1)

[0098] s y =(y m -y) / h; (1-2)

[0099] Among them, s x Indicates the horizontal axis offset ratio, s y Indicates the vertical axis offset ratio.

[0100] According to an embodiment of the present disclosure, the size offset ratio can be determined based on the size information of the disturbance region and the size information of the target detection frame, as shown in equations (1-3) and (1-4):

[0101] s w =w s / w; (1-3)

[0102] s h =h s / h; (1-4)

[0103] Among them, s w Indicates the length offset ratio, s h Indicates the width offset ratio.

[0104] According to an embodiment of the present disclosure, the object detection frame information of the first sample image may include the coordinate information (x1, y1) of the upper left corner of the object detection frame of the first sample image and the size information of the object detection frame of the first sample image. The size information of the object detection frame may include the length information w1 and the width information h1 of the object detection frame. Therefore, the object detection frame information of the first sample image can be expressed as (x1, y1, w1, h1).

[0105] According to an embodiment of the present disclosure, the disturbance information of the first sample image may include position information of the disturbance region of the first sample image and size information of the disturbance region of the first sample image.

[0106] According to an embodiment of the present disclosure, the position information of the disturbed area of the first sample image can be determined according to the position offset ratio and the position information and size information of the target detection frame of the first sample image, as shown in equations (2-1) and (2-2):

[0107] t x =0.5-(w1×s x+x1) / w1; (2-1)

[0108] t y =0.5-(h1×s y +y1) / h1; (2-2)

[0109] Wherein, tx represents the horizontal coordinate information of the upper left corner of the disturbance area of the first sample image; t y Indicates the vertical coordinate information of the upper left corner of the disturbance area of the first sample image.

[0110] According to an embodiment of the present disclosure, the size information of the disturbance region of the first sample image can be determined according to the size offset ratio and the size information of the target detection frame of the first sample image, as shown in equations (2-3) and (2-4):

[0111] t w =w1×s w ;

[0112] t h =h1×s h ;

[0113] For example, the target detection frame information of the target sample image can be (1, 2, 1, 2), and the perturbation information of the target sample image can be (4, 6, 1, 2). According to formulas (1-1) to (1-4), the horizontal coordinate offset ratio s can be obtained x The horizontal coordinate is 3, the vertical coordinate offset ratio is 2, the length offset ratio is 1, and the width offset ratio is 1. Taking the target detection frame information of the first sample image as (1, 1, 1, 1) as an example, according to formulas (2-1) to (2-4), the horizontal coordinate of the disturbed area of the first sample image is -3.5, the vertical coordinate is -2.5, the length is 1, and the width is 1. The disturbance information of the first sample image can be expressed as (-3.5, -2.5, 1, 1).

[0114] According to an embodiment of the present disclosure, perturbation information is added to the target sample image based on the perturbation information (4, 6, 1, 2) of the target sample image; and perturbation information is added to the first sample image based on the perturbation information (-3.5, -2.5, 1, 1) of the first sample image, so as to obtain a candidate adversarial image set.

[0115] For example, the perturbation region parameters are initialized based on the candidate position and candidate size of the perturbation region to obtain candidate perturbation region parameters. Based on the candidate perturbation region parameters, all sample images in the sample image set are scaled and translated according to the perturbation offset information to obtain a candidate adversarial image set after adding the perturbation information.

[0116] Figure 5 A schematic diagram of obtaining a candidate adversarial image set according to an embodiment of the present disclosure is schematically shown.

[0117] like Figure 5 As shown, in embodiment 500, perturbation information 5303 of the target sample image is obtained based on the m-th candidate position information 5301 and the s-th candidate size information 5302. Perturbation offset information 5305 is then obtained based on the perturbation information 5303 of the target sample image and the target detection frame information 5304 of the target sample information. Perturbation information 5307 of the first sample image is obtained based on the perturbation offset information 5305 and the target detection frame information 5306 of the first sample image. Finally, the perturbation information 5303 of the target sample image is added to the target sample image 5308, and the perturbation information 5307 of the first sample image is added to the first sample image 5309, to obtain a candidate adversarial image set 5310.

[0118] According to an embodiment of the present disclosure, the above operation S240 may include the following operations:

[0119] Perform object detection on the candidate adversarial image set to obtain adversarial image features and adversarial image confidences. Perform object detection on the sample image set to obtain sample image features and sample image confidences. Based on the target loss function, the image attack results are obtained based on the adversarial image features, adversarial images, sample image features, and sample image confidences. Based on the image attack results, a target adversarial image is obtained from the candidate adversarial image set.

[0120] According to an embodiment of the present disclosure, the target loss function may include a feature loss function and a confidence loss function.

[0121] According to an embodiment of the present disclosure, the feature loss function can be based on the sample image features and adversarial image features output by the feature layer of the target detection model, and the sample image and adversarial image features are averaged and pooled respectively, and the pooled results are obtained by calculating the mean square error. As shown in formula (3):

[0122] L1(X i , X i ′)=mse(feat1, feat2) (3)

[0123] Among them, L1(X i , X i ′) represents the feature loss function, feat1 represents the feature obtained by averaging the sample image features, and feat2 represents the feature obtained by averaging the adversarial image features; X i represents the sample image, X i ′ represents the candidate adversarial image obtained after adding perturbation information to the sample image.

[0124] According to an embodiment of the present disclosure, the confidence loss function can be obtained based on the confidence of the target detection box of the sample image and the confidence of the detection category, as shown in formula (4):

[0125] L2(X i )=sigmid(F(X i )[:,:,5:85])[:,:,t]*F(X i )[:,:,4:5] (4)

[0126] Among them, L2(X i ) represents the confidence loss function, F(X i ) represents the target detection result output, F(X i ) outputs a three-dimensional tensor. The first dimension of the tensor represents the number of sample images passed to the model for training at a single time. The second dimension of the tensor predicts the box. The third dimension of the tensor represents the location information and category information of each prediction box. T represents the label of the target object. F(X i )The total number of output tensors is 85. Among the 85 tensors, the first 4 tensors represent the location information of the prediction box, the fifth tensor represents the confidence of the target object in the prediction box, and the last 80 tensors represent the probability of the category to which the target object belongs.

[0127] According to an embodiment of the present disclosure, the objective loss function may be expressed as follows:

[0128] L(X i , X i ′)=0.1*L1(X i , X i ′)+L2(X i ) (5)

[0129] Figure 6 A schematic diagram of obtaining a target adversarial image according to an embodiment of the present disclosure is schematically shown.

[0130] like Figure 6 As shown, in embodiment 600, target detection is performed on a sample image set 641 to obtain sample image features 643 and sample image confidence 644. Target detection is performed on a candidate adversarial image set 642 to obtain adversarial image features 645 and adversarial image confidence 646. The sample image features 643, the sample image confidence 644, the adversarial image features 645, and the adversarial image confidence 646 are input into the target loss function shown in formula (5) to obtain the image attack result.

[0131] According to an embodiment of the present disclosure, candidate adversarial images and corresponding perturbation information can be learned by minimizing a target loss function. When the generated candidate adversarial image correctly identifies the target object with a confidence level less than 0.25, the image attack result can be considered a successful attack. The candidate adversarial image can then be determined as the target adversarial image 648.

[0132] According to the embodiments of the present disclosure, the target loss function constructed based on image features and confidence can realize the joint screening of target adversarial images from the two dimensions of image feature extraction and target recognition, so that the generation of target adversarial images no longer depends on the relationship between sample images, and the position and size of the disturbance area can be adaptively selected, so that the success rate of image attack is effectively improved.

[0133] According to the embodiments of the present disclosure, image attack testing based on adversarial images can effectively evaluate the potential recognition risks of the target detection model.

[0134] According to an embodiment of the present disclosure, the above method may further include the following operations:

[0135] The target adversarial image is detected using the target detection model to obtain a third detection result, which represents the confidence level that the target object is correctly identified. If the third detection result is less than a predetermined threshold, it is determined that the target detection model has an identification risk.

[0136] For example, the predetermined threshold could be 0.2. When a target adversarial image is fed into the target detection model, the output confidence score for correct identification of the target object is 0.1, indicating a successful image attack. If the target object in the adversarial image is occluded, the target detection model may be unable to identify the target object or may misclassify it, indicating a successful image attack and a recognition risk for the target detection model.

[0137] According to an embodiment of the present disclosure, in order to reduce the recognition risk of the target detection model and improve the robustness of the target detection model, a new sample image set can be combined with a target adversarial image and input into the target detection model for training. Based on the output target object category and target detection frame, a loss value is obtained based on the cross-entropy loss function. The model parameters of the target detection model are then adjusted based on the loss value so that the loss value gradually converges. When the convergence condition is reached, a target detection model with high robustness is obtained.

[0138] It should be noted that the training method for the above-mentioned object detection model can be supervised, using the target object category and location in the new sample image set as labels. Since supervised training of object detection models is a relatively mature technology, it will not be described in detail here.

[0139] Figure 7 The block diagram of the adversarial image generating device according to an embodiment of the present disclosure is schematically shown.

[0140] like Figure 7 As shown, the adversarial image generating device 700 of this embodiment includes a first processing module 710 , a second processing module 720 , a first obtaining module 730 and a second obtaining module 740 .

[0141] The first processing module 710 is configured to perform occlusion processing on a target sample image in the sample image set using the first patch image to obtain a plurality of first images having different disturbance region positions; and to obtain M second images and a plurality of candidate position information of disturbance regions corresponding to the M second images by performing target detection on the plurality of first images, where M is an integer greater than or equal to 1.

[0142] The second processing module 720 is used to adjust the size of the first patch image for the mth second image to obtain multiple third images with different sizes of disturbed regions; and determine multiple candidate sizes of the disturbed region by performing target detection on the multiple third images, where m is an integer greater than or equal to 1 and less than or equal to M.

[0143] The first obtaining module 730 is configured to add disturbance information to the sample images in the sample image set based on the plurality of candidate position information and the plurality of candidate sizes to obtain a candidate adversarial image set.

[0144] The second obtaining module 740 is configured to obtain a target adversarial image from the candidate adversarial image set by performing target detection on the candidate adversarial image set and the sample image set.

[0145] According to an embodiment of the present disclosure, the first processing module 710 includes a first determining submodule and a first obtaining submodule. The first determining submodule is configured to determine first starting position information of a first patch image based on position information of a target object in a target sample image. The first obtaining submodule is configured to perform a sliding window operation on the target sample image using the first patch image with a first step length based on the first starting position information to obtain multiple first images.

[0146] According to an embodiment of the present disclosure, the first obtaining submodule includes a first determining unit and a first obtaining unit. The first determining unit is configured to determine information about the first step length and the first moving direction. The first obtaining unit is configured to use the first patch image to perform a sliding window operation on the target sample image along the first moving direction with the first step length to obtain multiple first images.

[0147] According to an embodiment of the present disclosure, the first processing module includes a first detection submodule, a second acquisition submodule, and a second determination submodule. The first detection submodule is used to perform target detection on multiple first images to obtain a first detection result, wherein the first detection result represents the confidence that the target object is correctly identified. The second acquisition submodule is used to obtain M second images from the multiple first images based on the first detection result. The second determination submodule is used to determine the mth candidate position information of the disturbance area based on the position information of the first patch image in the mth second image for the mth second image.

[0148] According to an embodiment of the present disclosure, the second processing module 720 includes a third determining submodule and a first adjusting submodule. The third determining submodule is configured to determine second starting position information for resizing the first patch image based on the position information of the first patch image in the mth second image. The first adjusting submodule is configured to adjust the size of the first patch image with a second step size based on the second starting position information to obtain multiple second patch images.

[0149] According to an embodiment of the present disclosure, the first adjustment submodule includes: a second determination unit and an adjustment unit. The second determination unit is configured to determine information about a second step size and a second movement direction. The adjustment unit is configured to adjust the size of the first patch image along the second movement direction with a second step size to obtain a plurality of second patch images.

[0150] According to an embodiment of the present disclosure, the second processing module 720 includes a second detection submodule, a third acquisition submodule and a fourth determination submodule. Among them, the second detection submodule is used to perform target detection on multiple third images to obtain a second detection result, wherein the second detection result represents the confidence that the target object is correctly identified. The third acquisition submodule is used to obtain N fourth images from multiple third images based on the second detection result, where N is an integer greater than or equal to 1. The fourth determination submodule is used to determine the nth candidate size information of the disturbance area based on the size information of the second patch image in the nth fourth image, where n is an integer greater than or equal to 1 and less than or equal to N,

[0151] According to an embodiment of the present disclosure, a plurality of candidate position information includes M pieces, and candidate size information includes S pieces, wherein M and S are both integers greater than or equal to 1, and the first obtaining module includes: a fourth obtaining submodule, a fifth obtaining submodule, and a sixth obtaining submodule. The fourth obtaining submodule is used to obtain the perturbation information of the target sample image based on the mth candidate position information and the sth candidate size information, wherein s is an integer greater than or equal to 1 and less than or equal to S. The fifth obtaining submodule is used to obtain the perturbation information of the first sample image based on the perturbation information of the target sample image and the target detection frame information of the first sample image, wherein the first sample image represents any sample image in the sample image set except the target sample image. The sixth obtaining submodule is used to add the perturbation information of the target sample image to the target sample image, and add the perturbation information of the first sample image to the first sample image, to obtain a candidate adversarial image set.

[0152] According to an embodiment of the present disclosure, the fifth obtaining submodule includes a second obtaining unit and a third obtaining unit. The second obtaining unit is configured to obtain perturbation offset information based on the perturbation information of the target sample image and the target detection frame information of the target sample image. The third obtaining unit is configured to obtain perturbation information of the first sample image based on the perturbation offset information and the target detection frame information of the first sample image.

[0153] According to an embodiment of the present disclosure, the second acquisition module 740 includes a third detection submodule, a fourth detection submodule, a seventh acquisition submodule, and an eighth acquisition submodule. Among them, the third detection submodule is used to perform target detection on the candidate adversarial image set to obtain adversarial image features and the confidence of the adversarial image. The fourth detection submodule is used to perform target detection on the sample image set to obtain sample image features and the confidence of the sample image. The seventh acquisition submodule is used to obtain the image attack result based on the target loss function according to the adversarial image features, the adversarial image, the sample image features, and the confidence of the sample image. The eighth acquisition submodule is used to obtain the target adversarial image from the candidate adversarial image set based on the image attack result.

[0154] According to an embodiment of the present disclosure, the apparatus is further configured to detect the target adversarial image using the target detection model to obtain a third detection result, wherein the third detection result represents a confidence level that the target object has been correctly identified. If the third detection result is less than a predetermined threshold, determining that the target detection model has an identification risk.

[0155] Another aspect of the embodiments of the present disclosure further discloses a training device for a target detection model, including a training module for training a preset model using sample images and target adversarial images to obtain a target detection model, wherein the target adversarial image is obtained using the above-mentioned adversarial image generation method.

[0156] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0157] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0158] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method described above.

[0159] According to an embodiment of the present disclosure, a computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the method described above.

[0160] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0161] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0162] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0163] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the adversarial image generation method. For example, in some embodiments, the adversarial image generation method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the adversarial image generation method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the adversarial image generation method by any other appropriate means (e.g., by means of firmware).

[0164] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0165] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0166] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0167] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0168] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0169] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0170] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions provided by this disclosure can be achieved. This is not limited herein.

[0171] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for generating an adversarial image, comprising: Using the first patch image to perform occlusion processing on the target sample image in the sample image set, to obtain a plurality of first images having different disturbance area positions; Performing target detection on the plurality of first images to obtain a first detection result, wherein the first detection result represents a confidence level that the target object is correctly identified; Obtaining M second images from the plurality of first images according to the first detection result; For the mth second image, determining the mth candidate position information of the disturbance region according to the position information of the first patch image in the mth second image; wherein M is an integer greater than or equal to 1; Determining second starting position information for resizing the first patch image according to position information of the first patch image in the mth second image; determining information of a second step length and a second moving direction; adjusting the size of the first patch image along the second moving direction with the second step length to obtain a plurality of third images; Determine a plurality of candidate sizes of the disturbance region by performing target detection on the plurality of third images, where m is an integer greater than or equal to 1 and less than or equal to M; Adding disturbance information to the sample images in the sample image set based on the plurality of candidate position information and the plurality of candidate sizes to obtain a candidate adversarial image set; and By performing target detection on the candidate adversarial image set and the sample image set, a target adversarial image is obtained from the candidate adversarial image set.

2. The method according to claim 1, wherein The method of performing occlusion processing on the target sample image in the sample image set by using the first patch image to obtain a plurality of first images having different disturbance area positions includes: determining first starting position information of the first patch image according to position information of the target object in the target sample image; and According to the first starting position information, a sliding window operation is performed on the target sample image using the first patch image with a first step length to obtain the multiple first images.

3. The method according to claim 2, wherein: The step of performing a sliding window operation on the target sample image using the first patch image with a first step length according to the first starting position information to obtain the plurality of first images includes: Information determining the first step length and first movement direction; and Using the first patch image, a sliding window operation is performed on the target sample image along the first moving direction with the first step length to obtain the multiple first images.

4. The method according to claim 1, wherein The determining a plurality of candidate sizes of the disturbance region by performing target detection on the plurality of third images comprises: Performing target detection on the plurality of third images to obtain a second detection result, wherein the second detection result represents a confidence level that the target object is correctly identified; Obtaining N fourth images from the plurality of third images according to the second detection result, where N is an integer greater than or equal to 1; and For the nth fourth image, determine nth candidate size information of the disturbance area according to the size information of the second patch image in the nth fourth image, where n is an integer greater than or equal to 1 and less than or equal to N.

5. The method according to claim 1, wherein The plurality of candidate position information includes M pieces, and the candidate size information includes S pieces, where M and S are both integers greater than or equal to 1. Adding disturbance information to sample images in the sample image set based on the plurality of candidate position information and the plurality of candidate sizes to obtain a candidate adversarial image set includes: Obtaining disturbance information of the target sample image according to the mth candidate position information and the sth candidate size information, where s is an integer greater than or equal to 1 and less than or equal to S; Obtaining disturbance information of the first sample image according to the disturbance information of the target sample image and target detection frame information of the first sample image, where the first sample image represents any sample image in the sample image set except the target sample image; and The perturbation information of the target sample image is added to the target sample image, and the perturbation information of the first sample image is added to the first sample image to obtain a candidate adversarial image set.

6. The method according to claim 5, wherein: The obtaining the disturbance information of the first sample image according to the disturbance information of the target sample image and the target detection frame information of the first sample image includes: Obtaining disturbance offset information according to the disturbance information of the target sample image and the target detection frame information of the target sample image; and Perturbation information of the first sample image is obtained according to the disturbance offset information and the target detection frame information of the first sample image.

7. The method according to claim 1, wherein The step of performing target detection on the candidate adversarial image set and the sample image set to obtain a target adversarial image from the candidate adversarial image set includes: Performing target detection on the candidate adversarial image set to obtain adversarial image features and adversarial image confidences; Performing target detection on the sample image set to obtain sample image features and confidence levels of the sample images; Obtaining an image attack result based on the target loss function and the adversarial image features, the adversarial image, the sample image features, and the confidence of the sample image; and According to the image attack result, a target adversarial image is obtained from the candidate adversarial image set.

8. The method according to claim 1, wherein The target adversarial image is used to evaluate the recognition risk of the target detection model.

9. The method according to claim 1, wherein The method further comprises: Detecting the target adversarial image using the target detection model to obtain a third detection result, where the third detection result represents a confidence level that the target object is correctly identified; When the third detection result is less than a predetermined threshold, it is determined that the target detection model has an identification risk.

10. A device for generating an adversarial image, comprising: a first processing module, configured to perform occlusion processing on a target sample image in the sample image set using the first patch image, to obtain a plurality of first images having different disturbance region positions; Performing target detection on the multiple first images to obtain a first detection result, wherein the first detection result represents a confidence level that the target object is correctly identified; obtaining M second images from the multiple first images based on the first detection result; and determining, for the mth second image, an mth candidate position information of the disturbance region based on position information of the first patch image in the mth second image, wherein M is an integer greater than or equal to 1; a second processing module, configured to determine, based on the position information of the first patch image in the mth second image, second starting position information for resizing the first patch image; determine information about a second step size and a second movement direction; adjust the size of the first patch image along the second movement direction with the second step size to obtain a plurality of third images; and determine a plurality of candidate sizes of the disturbance region by performing target detection on the plurality of third images, where m is an integer greater than or equal to 1 and less than or equal to M; A first obtaining module is configured to add disturbance information to the sample images in the sample image set based on the plurality of candidate position information and the plurality of candidate sizes to obtain a candidate adversarial image set; and The second obtaining module is configured to obtain a target adversarial image from the candidate adversarial image set by performing target detection on the candidate adversarial image set and the sample image set.

11. The device according to claim 10, wherein The first processing module includes: A first determining submodule is configured to determine first starting position information of the first patch image according to position information of the target object in the target sample image; and The first obtaining submodule is configured to perform a sliding window operation on the target sample image using the first patch image with a first step length according to the first starting position information to obtain the multiple first images.

12. The device according to claim 11, wherein The first obtaining submodule includes: A first determining unit, configured to determine information of a first step length and a first moving direction; and The first obtaining unit is configured to use the first patch image to perform a sliding window operation on the target sample image along the first moving direction with the first step length to obtain the multiple first images.

13. The device according to claim 10, wherein The second processing module includes: a second detection submodule, configured to perform target detection on the plurality of third images to obtain a second detection result, wherein the second detection result represents a confidence level that the target object is correctly identified; a third obtaining submodule, configured to obtain N fourth images from the plurality of third images according to the second detection result, where N is an integer greater than or equal to 1; and The fourth determination submodule is used to determine, for the nth fourth image, the nth candidate size information of the disturbance area according to the size information of the second patch image in the nth fourth image, where n is an integer greater than or equal to 1 and less than or equal to N.

14. The device according to claim 10, wherein The plurality of candidate position information includes M pieces, and the plurality of candidate size information includes S pieces, where M and S are both integers greater than or equal to 1, and the first obtaining module includes: a fourth obtaining submodule, configured to obtain disturbance information of the target sample image according to the mth candidate position information and the sth candidate size information, where s is an integer greater than or equal to 1 and less than or equal to S; a fifth obtaining submodule, configured to obtain disturbance information of the first sample image based on the disturbance information of the target sample image and the target detection frame information of the first sample image, where the first sample image represents any sample image in the sample image set except the target sample image; and The sixth obtaining submodule is configured to add the perturbation information of the target sample image to the target sample image, and add the perturbation information of the first sample image to the first sample image, to obtain a candidate adversarial image set.

15. The device according to claim 10, wherein The fifth acquisition submodule includes: A second obtaining unit is configured to obtain disturbance offset information according to the disturbance information of the target sample image and the target detection frame information of the target sample image; and The third obtaining unit is configured to obtain disturbance information of the first sample image according to the disturbance offset information and the target detection frame information of the first sample image.

16. The device according to claim 10, wherein The second obtaining module includes: A third detection submodule is used to perform target detection on the candidate adversarial image set to obtain adversarial image features and adversarial image confidence; a fourth detection submodule, configured to perform target detection on the sample image set to obtain sample image features and confidence levels of the sample images; a seventh obtaining submodule, configured to obtain an image attack result based on a target loss function, the adversarial image feature, the adversarial image, the sample image feature, and the confidence of the sample image; and An eighth obtaining submodule is configured to obtain a target adversarial image from the candidate adversarial image set according to the image attack result.

17. The device according to claim 10, wherein The target adversarial image is used to evaluate the recognition risk of the target detection model.

18. The device according to claim 17, wherein The device is also used for: Detecting the target adversarial image using the target detection model to obtain a third detection result, where the third detection result represents a confidence level that the target object is correctly identified; When the third detection result is less than a predetermined threshold, it is determined that the target detection model has an identification risk.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images