Adversarial sample generation method and system for robustness testing of sign recognition systems

Abstract

The present invention discloses an adversarial sample generation method and system for robustness testing of a sign recognition system, the method comprising: transforming a standard sign image based on an image transformer and transformation parameters to obtain a first image; embedding the first image into a background image to obtain a second image; performing target detection on the second image based on a target detection model to obtain a plurality of bounding boxes; screening out a target bounding box from a plurality of bounding boxes; selecting a corresponding loss function to process the target bounding box to obtain a loss value; updating the current transformation parameters according to the loss value and using an optimization method, and iteratively processing the standard sign image based on the updated transformation parameters and the above process. The above adversarial sample generation method can quickly generate a variety of adversarial samples, so that the generated adversarial samples are closer to the actual application environment, which is helpful to test the robustness of the system in actual operation.

Description

Technical Field

[0001] The present invention relates to the field of software testing technology, and in particular to a method and system for generating adversarial samples for robustness testing of a sign recognition system. Background Art

[0002] With the rapid development of intelligent connected vehicles and autonomous driving technologies, traffic sign recognition systems have become one of the key technologies to ensure vehicle driving safety and realize autonomous driving functions. These systems mainly rely on computer vision and deep learning technologies to capture and analyze road images in real time through cameras to identify traffic signs and assist driving decisions. However, as technology becomes increasingly advanced, security risks also arise, and one of the most challenging issues is the threat of adversarial sample attacks.

[0003] Adversarial examples refer to small and carefully designed perturbations imposed on the original input data, which cause the deep learning model to produce misjudged inputs. This attack method can effectively confuse the deep learning model and cause it to misidentify or interpret traffic signs. In traffic sign recognition systems, such misjudgments may cause vehicles to fail to correctly recognize important traffic signs, such as misidentifying a "stop" sign as a "speed limit" sign, thereby causing serious traffic accident risks.

[0004] Therefore, adversarial sample attacks have important research significance for the robustness test of traffic sign recognition systems. First, by studying and evaluating the performance of traffic sign recognition systems under adversarial sample attacks, the potential vulnerabilities of the system can be revealed, thereby providing a reference for improving the security design of the system. Second, through effective adversarial sample generation and detection mechanisms, the reliability and security of the system in harsh environments can be significantly improved.

[0005] In this context, how to efficiently generate adversarial samples to improve the efficiency and coverage of testing has become a key research direction. An effective adversarial sample generation method can not only reveal the weaknesses of the system, but also lay the foundation for the development of a more robust and secure traffic sign recognition system. Through such research, we can provide more reliable technical support for autonomous driving vehicles in complex road environments. Summary of the invention

[0006] The object of the present invention is to provide an adversarial sample generation method and system for robustness testing of a sign recognition system, which can quickly generate high-quality adversarial samples.

[0007] In order to achieve the above object, the present invention provides an adversarial sample generation method for robustness testing of a sign recognition system, which comprises:

[0008] Obtain a standard logo image with characteristic logos;

[0009] Transforming the standard marker image based on an image transformer and transformation parameters to obtain a first image with an anti-disturbance factor;

[0010] embedding the first image into a randomly selected background image to obtain a second image, wherein the second image shows an actual usage scenario of the sign represented by the standard sign image;

[0011] performing object detection on the second image based on an object detection model used by the landmark recognition system to be tested to obtain a plurality of bounding boxes and corresponding prediction information;

[0012] According to the attack target, a filter is selected to filter the plurality of said bounding boxes to obtain a target bounding box;

[0013] According to the attack mode selected by the current test, a corresponding loss function is selected to process the target bounding box to obtain a loss value;

[0014] According to the loss value, the current transformation parameters are updated by an optimization method, and the standard logo image is iteratively processed based on the updated transformation parameters until an iteration end condition is met, and the second image finally generated is output as an adversarial sample.

[0015] Preferably, the method of transforming the standard logo image includes any one or more of random erasing transformation, horizontal flipping transformation, hue transformation, saturation transformation, gamma transformation, and contrast transformation.

[0016] Preferably, a back propagation mechanism and a stochastic gradient descent algorithm are used to optimize and update the transformation parameters.

[0017] Preferably, the method for selecting the loss function includes:

[0018] When the attack mode is a target attack, the loss function is selected based on maximizing the confidence of the target category and minimizing the confidence of other categories outside the target category and the image disturbance amount;

[0019] When the attack mode is a hidden attack, the loss function is selected based on minimizing the target category confidence and minimizing the image disturbance amount;

[0020] When the attack mode is appearance attack, the loss function is selected based on maximizing the target category confidence and minimizing the image disturbance amount;

[0021] When the attack mode is a non-target attack, the loss function is selected based on maximizing the confidence of the feature mark while minimizing the target category confidence and the image disturbance amount;

[0022] The target category is a preset error category.

[0023] Preferably, the characteristic sign is a road traffic sign.

[0024] Preferably, when the transformation process of the standard logo image is performed for the first time, the transformation parameters are randomly generated based on a preset parameter value interval.

[0025] Preferably, any one of a preset first threshold value related to the loss value and a preset second threshold value related to the number of iterations is used as the iteration end condition.

[0026] The present invention also provides an adversarial sample generation system for robustness testing of a sign recognition system, characterized in that it includes a data processor, which generates adversarial samples based on the adversarial sample generation method as described above.

[0027] The present invention also provides an adversarial sample generation system, which includes:

[0028] one or more processors;

[0029] Memory;

[0030] and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs include instructions for executing the adversarial sample generation method as described above.

[0031] The present invention also provides a computer-readable storage medium, characterized in that it includes a computer program, and the computer program can be executed by a processor to complete the adversarial sample generation method as described above.

[0032] Compared with the prior art, the adversarial sample generation method disclosed in the above technical solution uses an image transformer and an optimization algorithm to iteratively update the transformation parameters, and can quickly generate a variety of adversarial samples. This not only improves the efficiency of adversarial sample generation, but also enables the generated adversarial samples to cover a wider range of potential attack modes; in addition, the transformed image is embedded in a random background to simulate a real scene, so that the generated adversarial samples are closer to the actual application environment, which helps to test the robustness of the system in actual operation; furthermore, the adaptive bounding box filtering mechanism can dynamically screen out the target bounding box that meets the attack requirements, ensuring the accuracy and pertinence when generating adversarial perturbations. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flow chart of the adversarial sample generation method in an embodiment of the present invention.

[0034] Figure 2 Schematic diagram of the principle of the adversarial sample generation method in an embodiment of the invention.

[0035] Figure 3 Schematic diagram of image transformation in an embodiment of the present invention.

[0036] Figure 4 FIG. 4 is a schematic diagram of image embedding in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to explain the technical content, structural features, achieved objectives and effects of the present invention in detail, the following is a detailed description in conjunction with the implementation methods and the accompanying drawings.

[0038] This embodiment discloses an adversarial sample generation method for use in generating adversarial samples during a robustness test of a sign recognition system. For example, for a traffic sign recognition system in an intelligent driving system, a robustness test is required to verify the performance of the traffic sign recognition system, and adversarial samples are required during the test.

[0039] Adversarial sample = original image + adversarial perturbation. Adversarial perturbation is the key factor in generating adversarial samples, which gives the original image the ability to deceive the model.

[0040] Adversarial perturbations are small, carefully designed noises or changes added to the original input image (for example, an image of a traffic sign that says "Speed ​​Limit 40"). This perturbation itself may be imperceptible to the naked eye, but it can fool the deep learning model into making incorrect recognition results for the perturbed image.

[0041] like Figure 1 and Figure 2 , the adversarial sample generation method includes the following steps:

[0042] S10: Acquire a standard sign image with a characteristic mark, for example, the characteristic mark is a traffic sign of “Speed ​​Limit 40”.

[0043] S11: transforming the standard logo image based on the image transformer and the transformation parameters to obtain a first image with an anti-disturbance factor (such as Figure 3 ). Specifically, the standard logo image is subjected to illumination transformation, angle transformation, and resolution transformation, for example, the brightness of the standard logo image is increased by 10%, the image is rotated by 5°, the resolution is reduced by 5%, and a slight blur is introduced.

[0044] S12: Embed the first image into a randomly selected background image to obtain a second image (such as Figure 4 ), the second image shows the actual usage scenario of the sign represented by the standard sign image. For example, the standard sign image of "Speed ​​Limit 40" is randomly embedded in any position of the background image of the real road environment, thereby simulating the actual usage environment state of the characteristic sign.

[0045] S13: Performing target detection on the second image based on the target detection model used by the landmark recognition system to be tested (such as YOLO v5) to obtain a plurality of bounding boxes and corresponding prediction information, where the prediction information includes confidence levels, category probabilities, etc. corresponding to the bounding boxes.

[0046] S14: According to the attack target, select a filter to filter several bounding boxes to obtain the target bounding box. YOLOv5 may predict multiple bounding boxes, some of which may be "speed limit 40", some may be other objects, or even some are misjudgments. Therefore, it is necessary to use filters and select different strategies to filter the target bounding box according to different attack targets.

[0047] S15: According to the attack mode selected by the current test, the corresponding loss function is selected to process the target bounding box to obtain a loss value. The loss value represents the gap between the currently generated adversarial sample (i.e., the second image) and the attack target. By continuously adjusting the adversarial perturbation, the loss value can be minimized, thereby generating an adversarial sample that can effectively deceive the target traffic sign recognition system.

[0048] S16: Therefore, according to the loss value, the current transformation parameters are updated by using a preset optimization method. Specifically, the back propagation mechanism and the stochastic gradient descent algorithm (SGD) are used to optimize and update the transformation parameters.

[0049] S17: Whether the iteration end condition is met, if not, return to step S11, if yes, enter S18.

[0050] S18: Output the finally generated second image as an adversarial sample.

[0051] It should be noted that when the transformation process of the standard logo image is performed for the first time, the transformation parameters are randomly generated based on the preset parameter value range.

[0052] On the other hand, any one of a preset first threshold value related to the loss value and a second threshold value related to the number of iterations is used as the iteration end condition.

[0053] Specifically, when the loss function value reaches a preset first threshold or no longer decreases significantly, the iteration can be stopped, which means that the adversarial perturbation has achieved a certain degree of optimization effect, and the benefits of continuing the iteration are not great.

[0054] In addition, setting the maximum number of iterations (the second threshold) as the iteration end condition can prevent the iteration process from going on indefinitely.

[0055] It should be noted that these two iteration end conditions can be set at the same time, or one of them can be selected. When they are set at the same time, the one that is satisfied first will prevail, that is, no matter which end condition is satisfied first, the current iteration process will end.

[0056] Furthermore, the amplitude of the adversarial perturbation can be used as the condition for ending the iteration. In order to ensure the imperceptibility of adversarial samples, the maximum amplitude of the adversarial perturbation can be limited. When the perturbation amplitude reaches the upper limit, the iteration should be stopped even if the loss function value is still decreasing.

[0057] On the other hand, the method of transforming the standard logo image includes any one or more of random erasing transformation, horizontal flipping transformation, hue transformation, saturation transformation, gamma transformation, and contrast transformation.

[0058] It should be noted that the transformation parameter does not refer to a single parameter, but a group of parameters, which can be represented in the form of a vector or a tensor, and its dimension depends on the number of transformation methods used and the number of parameters of each method.

[0059] The above adversarial sample generation method first performs various transformations on the standard sign image (simulating changes in lighting, angle, etc.), and then embeds the transformed image into the background of the real driving scene. Next, the target detection model predicts the embedded image and calculates the error based on the preset loss function. Finally, the back propagation and stochastic gradient descent algorithms are used to adjust the adversarial perturbation so that it can maximize the error rate of the target detection model. This process is iterated until an adversarial perturbation that can effectively deceive the model is generated. The standard sign image with the adversarial perturbation is the required adversarial sample.

[0060] Therefore, for this adversarial sample generation method, firstly, the image transformer and optimization algorithm are used to iteratively update the transformation parameters, so that a variety of adversarial samples can be quickly generated. This not only improves the efficiency of adversarial sample generation, but also makes the generated adversarial samples cover a wider range of potential attack modes; secondly, the transformed image is embedded in a random background to simulate the real scene, so that the generated adversarial samples are closer to the actual application environment, which helps to test the robustness of the system in actual operation; in addition, the adaptive bounding box filtering mechanism can dynamically screen out the target bounding box that meets the attack requirements, ensuring the accuracy and pertinence when generating adversarial perturbations.

[0061] On the other hand, the selection methods of loss function include:

[0062] When the attack mode is a target attack, the loss function is selected based on maximizing the confidence of the target category while minimizing the confidence of other categories outside the target category and the amount of image perturbation;

[0063] When the attack mode is hidden attack, the loss function is selected based on minimizing the target category confidence and minimizing the image perturbation amount;

[0064] When the attack mode is appearance attack, the loss function is selected based on maximizing the target category confidence and minimizing the image perturbation amount;

[0065] When the attack mode is non-target attack, the loss function is selected based on maximizing the confidence of the feature mark while minimizing the target category confidence and the image perturbation amount;

[0066] The target category is the preset error category.

[0067] The following is a specific example to illustrate the specific implementation process of the adversarial sample generation method in an embodiment of the present invention.

[0068] 1. Image transformation:

[0069] Input a clear "Speed ​​Limit 40" standard sign image to the image converter.

[0070] The parameter generator in this image transformer randomly generates the following transformation parameters:

[0071] Lighting change: Brightness increased by 10%;

[0072] Angle transformation: rotate 5 degrees;

[0073] Resolution conversion: slightly reduce the resolution (for example, shrink by 5% and then enlarge back to the original size, introducing slight blur);

[0074] Distance Transform: simulates long-distance shooting, slightly blurring and reducing contrast;

[0075] Blur Transform: Applies a slight Gaussian blur;

[0076] Other random transformations: Select and apply one or more transformations such as random erase, horizontal flip, hue fine-tuning, etc. (For example, randomly select hue fine-tuning and adjust the hue parameter to randomly change within a certain range).

[0077] These transformations are applied sequentially to the standard logo image, based on the generated random parameters. Each transformation must be differentiable in order to back-propagate the gradients.

[0078] After various transformations, the "Speed ​​Limit 40" sign image is slightly deformed, and the lighting conditions are slightly different. The transformed "Speed ​​Limit 40" sign image is named Image A, as shown in Figure 3 .

[0079] 2. Image Embedding:

[0080] The transformed "Speed ​​Limit 40" sign image is embedded into a randomly selected real road background image, and the "Speed ​​Limit 40" sign image is randomly embedded into any position of the background image. A real road scene image containing the "Speed ​​Limit 40" sign is obtained, and the image is named image B, such as Figure 4 .

[0081] 3. Boundary filtering module:

[0082] Run the object detection model (YOLO v5) to perform object detection on image B and get all detected bounding boxes and their confidences.

[0083] Take the target attack (misjudging "speed limit 40" as "speed limit 60") as an example to select the filter, and select the S-BBOX filter. This filter selects the bounding box with the largest IOU with the embedded position from all bounding boxes as the target bounding box.

[0084] A filtered object bounding box representing what YOLO v5 thinks is the location and size of a “Speed ​​Limit 40” sign.

[0085] 4. Loss function processing:

[0086] The current attack mode is target attack, so select the loss function of target attack, which consists of three parts:

[0087] Maximize the confidence level of the "Speed ​​Limit 40" sign;

[0088] Maximize the probability of the "Speed ​​Limit 60" category;

[0089] Minimize the probability of all other classes (especially speed limit signs other than "Speed ​​Limit 40").

[0090] Feed the object bounding box into the chosen loss function to obtain the loss value.

[0091] 5. Training optimization:

[0092] The loss value output by the loss function is used as a parameter, the gradient is calculated using the back propagation algorithm, and the adversarial perturbation is updated using the SGD algorithm to obtain the updated transformation parameters.

[0093] 6. Iterative processing:

[0094] The original "Speed ​​Limit 40" sign image is transformed based on the updated transformation parameters, and the above process is re-executed. This iteration is repeated until the iteration end condition is met.

[0095] 7. Generate adversarial examples:

[0096] After the iteration, the newly generated image B is output to obtain an adversarial sample.

[0097] This adversarial example is visually very similar to the original image, but is able to fool the YOLO v5 model into misclassifying the sign as “Speed ​​Limit 60”.

[0098] In another preferred embodiment of the present invention, an adversarial sample generation system for robustness testing of a sign recognition system is disclosed, which includes a data processor, and the data processor generates adversarial samples based on the adversarial sample generation method in the above embodiment.

[0099] The present invention also discloses another adversarial sample generation system, which includes one or more processors, a memory and one or more programs, wherein one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the program includes instructions for executing the adversarial sample generation method as described above. The processor can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits for executing related programs to implement the functions required to be performed by the modules in the adversarial sample generation system of the embodiment of the present application, or to execute the adversarial sample generation method of the method embodiment of the present application.

[0100] The present invention also discloses a computer-readable storage medium, which includes a computer program, and the computer program can be executed by a processor to complete the adversarial sample generation method as described above. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a read-only memory (ROM), or a random access memory (RAM), or a magnetic medium, such as a floppy disk, a hard disk, a tape, a magnetic disk, or an optical medium, such as a digital versatile disc (DVD), or a semiconductor medium, such as a solid state disk (SSD), etc.

[0101] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the scope of the patent application of the present invention are still within the scope covered by the present invention.

Claims

1. A method for generating adversarial samples for robustness testing of a sign recognition system, characterized in that: include: Obtain a standard logo image with characteristic logos; Transforming the standard marker image based on an image transformer and transformation parameters to obtain a first image with an anti-disturbance factor; embedding the first image into a randomly selected background image to obtain a second image, wherein the second image shows an actual usage scenario of the sign represented by the standard sign image; performing object detection on the second image based on an object detection model used by the landmark recognition system to be tested to obtain a plurality of bounding boxes and corresponding prediction information; According to the attack target, a filter is selected to filter the plurality of said bounding boxes to obtain a target bounding box; According to the attack mode selected by the current test, a corresponding loss function is selected to process the target bounding box to obtain a loss value; According to the loss value, the current transformation parameters are updated by an optimization method, and the standard logo image is iteratively processed based on the updated transformation parameters until an iteration end condition is met, and the second image finally generated is output as an adversarial sample.

2. The adversarial sample generation method according to claim 1, characterized in that: The method of transforming the standard logo image includes any one or more of random erasing transformation, horizontal flipping transformation, hue transformation, saturation transformation, gamma transformation, and contrast transformation.

3. The adversarial sample generation method according to claim 1, characterized in that: The transformation parameters are optimized and updated using a back-propagation mechanism and a stochastic gradient descent algorithm.

4. The adversarial sample generation method according to claim 1, characterized in that: The method for selecting the loss function includes: When the attack mode is a target attack, the loss function is selected based on maximizing the confidence of the target category and minimizing the confidence of other categories outside the target category and the image disturbance amount; When the attack mode is a hidden attack, the loss function is selected based on minimizing the target category confidence and minimizing the image disturbance amount; When the attack mode is appearance attack, the loss function is selected based on maximizing the target category confidence and minimizing the image disturbance amount; When the attack mode is a non-target attack, the loss function is selected based on maximizing the confidence of the feature mark while minimizing the target category confidence and the image disturbance amount; The target category is a preset error category.

5. The adversarial sample generation method according to claim 1, characterized in that: The characteristic sign is a road traffic sign.

6. The adversarial sample generation method according to claim 1, characterized in that: When the transformation process of the standard logo image is performed for the first time, the transformation parameters are randomly generated based on a preset parameter value interval.

7. The adversarial sample generation method according to claim 1, characterized in that: Any one of a preset first threshold value related to the loss value and a preset second threshold value related to the number of iterations is used as an iteration end condition.

8. An adversarial sample generation system for robustness testing of a sign recognition system, characterized in that: It includes a data processor, which generates an adversarial sample based on the adversarial sample generation method according to any one of claims 1 to 7.

9. A system for generating adversarial samples, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs comprising instructions for executing the adversarial sample generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It includes a computer program, which can be executed by a processor to complete the adversarial sample generation method as described in any one of claims 1 to 7.

Images