Adversarial patch generation method and device, computer equipment and storage medium

Through data enhancement and color adjustment technology of adversarial patches, the problem of poor obstruction patches in the existing technology is solved, and more obscure adversarial patches are generated, which improves the attack effect in complex environments.

CN118967875APending Publication Date: 2024-11-15启元实验室
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410981427.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing adversarial patch attack methods ignore the concealment of adversarial patches, making adversarial patches easy to be discovered by humans and reducing the practicality of attacks.

Method used

By obtaining the initial adversarial patch and background images, adding interference information, optimizing the loss function, and adjusting the color of the adversarial patch using the color replacement matrix of the background image, generating target adversarial patches to improve its concealment.

Benefits of technology

It significantly improves the environmental concealment of the anti-patch, making it more difficult to detect in complex environment scenarios, and enhances the practicality of the attack.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967875A_ABST
    Figure CN118967875A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of adversarial attacks, and discloses an adversarial patch generation method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining interference information according to a preset mode, adding the interference information to an initial adversarial patch, and obtaining a first adversarial patch; adding the first adversarial patch to a target area in the original image to obtain a first image with the first adversarial patch; determining a target image according to the original image and the first image; the target image carries a second adversarial patch determined according to the first adversarial patch; optimizing the second adversarial patch through a loss function to obtain a third adversarial patch; and adjusting the color of each pixel point in the third adversarial patch according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch. According to the method, the main color in the background image is extracted to replace the color of the adversarial patch, so that the generation speed of the adversarial patch and the environment concealment are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of anti-attack technology, and in particular to an anti-attack patch generation method, device, computer equipment and storage medium. Background Art

[0002] With the continuous development of artificial intelligence technology, machine vision is widely used in modern intelligent systems, such as intelligent robots, face recognition, and autonomous driving. Machine vision acquires images through cameras and uses deep learning algorithms to detect and identify objects contained in the images, thereby achieving the purpose of perceiving the environment. The perception results of machine vision usually affect the decision-making of intelligent systems. In order to ensure the security of machine vision perception results, research on adversarial attack technologies has increased in recent years.

[0003] At present, there are two main types of adversarial attack technologies. One is algorithmic adversarial attack, which means that the attacker misleads the algorithm model by adding subtle perturbations to the input data. The other is physical attack, which means that the attacker does not directly manipulate the digital input, but manipulates the target object in the physical world to indirectly deceive the model. Adversarial patch is a commonly used physical attack method for intelligent algorithms. It achieves the effect of deceiving deep learning models by pasting adversarial patches onto the target. However, the existing adversarial patch attack method ignores the concealment of adversarial patches, making adversarial patches easy to be discovered by humans, reducing the practicality of the attack.

[0004] Therefore, how to generate an adversarial patch with better concealment becomes a technical problem that technical personnel in this field need to solve urgently. Summary of the invention

[0005] In view of this, the present invention provides a method, apparatus, computer device and storage medium for generating an adversarial patch to solve the problem of poor concealment of the adversarial patch in the related art.

[0006] In a first aspect, the present invention provides a method for generating an adversarial patch, comprising:

[0007] Get the initial adversarial patch, the original image, and the background image;

[0008] Acquire interference information according to a preset method, and add the interference information to the initial adversarial patch to obtain a first adversarial patch;

[0009] Adding the first adversarial patch to a target area in the original image to obtain a first image with the first adversarial patch; the target area is determined according to position information of an attacked target in a detection frame of the original image, and the detection frame includes the attacked target;

[0010] Determine a target image according to the original image and the first image with the first adversarial patch; the target image has a second adversarial patch determined according to the first adversarial patch;

[0011] Optimizing the second adversarial patch by using a loss function to obtain a third adversarial patch;

[0012] Determine a color replacement matrix of the background image, and determine a color probability matrix of the third adversarial patch; the color replacement matrix is ​​determined based on a preset number of colors that appear the most times among the colors of all pixels in the background image; the color probability matrix represents the probability of each color in the third adversarial patch;

[0013] The color of each pixel in the third adversarial patch is adjusted according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

[0014] In an optional implementation, determining the color replacement matrix of the background image includes:

[0015] Determine the colors of all pixels in the background image;

[0016] Determine a preset number of colors that appear the most times among the colors of all pixels in the background image;

[0017] The color replacement matrix of the background image is determined according to a preset number of colors that appear most frequently among the colors of all pixels in the background image.

[0018] In an optional implementation, adjusting the color of each pixel in the third adversarial patch according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain the target adversarial patch includes:

[0019] The color of each pixel in the third adversarial patch is adjusted according to the product of the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

[0020] In an optional implementation, adding the first adversarial patch to the target area in the original image to obtain a first image with the first adversarial patch includes:

[0021] The first adversarial patch is subjected to a preset transformation according to the position information of the attacked target in the detection frame of the original image and a preset ratio, and the first adversarial patch obtained after the preset transformation is added to the target area in the original image to obtain a first image with the first adversarial patch.

[0022] In an optional implementation, optimizing the second adversarial patch by using a loss function to obtain a third adversarial patch includes:

[0023] Inputting the target image into a pre-acquired target detection model to obtain a detection result; the detection result includes the confidence of the detection box in the target image;

[0024] Determine the loss corresponding to the detection result through a loss function;

[0025] The gradient of the second adversarial patch is optimized according to the loss corresponding to the detection result to obtain a third adversarial patch.

[0026] In an optional implementation, obtaining interference information according to a preset manner includes: generating contrast, brightness and noise by random sampling according to a uniform distribution.

[0027] In an optional implementation, after obtaining the target adversarial patch, the method further includes: testing the attack effect of the target adversarial patch.

[0028] In a second aspect, the present invention provides a device for generating an adversarial patch, comprising:

[0029] A first processing module is used to obtain an initial adversarial patch, an original image and a background image; obtain interference information according to a preset method, and add the interference information to the initial adversarial patch to obtain a first adversarial patch;

[0030] A second processing module, configured to add the first adversarial patch to a target area in the original image to obtain a first image with the first adversarial patch; the target area is determined according to position information of an attacked target in a detection frame of the original image, and the detection frame includes the attacked target;

[0031] A third processing module, configured to determine a target image according to the original image and the first image with the first adversarial patch; the target image has a second adversarial patch determined according to the first adversarial patch;

[0032] A fourth processing module, configured to optimize the second adversarial patch by using a loss function to obtain a third adversarial patch;

[0033] a fifth processing module, configured to determine a color replacement matrix of the background image and a color probability matrix of the third adversarial patch; the color replacement matrix is ​​determined based on a preset number of colors that appear the most frequently among the colors of all pixels in the background image; the color probability matrix represents the probability of each color in the third adversarial patch;

[0034] The sixth processing module is used to adjust the color of each pixel in the third adversarial patch according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

[0035] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the adversarial patch generation method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for generating an adversarial patch according to the first aspect or any corresponding embodiment thereof.

[0037] The technical solution provided by the present invention has the following technical effects:

[0038] The embodiment of the present invention can perform data enhancement on the initial adversarial patch by adding interference information to the initial adversarial patch, thereby increasing the complexity and diversity of the adversarial patch. The first adversarial patch is added to the target area in the original image to obtain a first image with the first adversarial patch, which can simulate the process of adversarial attack. The target image and the second adversarial patch determined according to the first adversarial patch in the target image can be obtained by fusing the original image and the first image with the first adversarial patch. The second adversarial patch is further optimized by the loss function to obtain the third adversarial patch. The color of each pixel in the third adversarial patch is adjusted according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain the target adversarial patch. The main color in the background image is extracted by the color replacement matrix of the background image, and the color of the adversarial patch is replaced, so that adversarial patches in different complex environmental scenes can be quickly generated, and the environmental concealment of the adversarial patch is significantly improved while ensuring the adversarial effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 is a schematic diagram of a process of generating an adversarial patch according to an embodiment of the present invention;

[0041] Figure 2is a schematic diagram of a third adversarial patch image according to an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram comparing the anti-camouflage and environmental concealment effects according to an embodiment of the present invention;

[0043] Figure 4 is a schematic diagram of a background image according to an embodiment of the present invention;

[0044] Figure 5 is a schematic diagram of a color replacement matrix according to an embodiment of the present invention;

[0045] Figure 6 is a schematic diagram of a target adversarial patch image according to an embodiment of the present invention;

[0046] Figure 7 is a schematic diagram of the overall process of the method for generating an adversarial patch according to an embodiment of the present invention;

[0047] Figure 8 is a structural block diagram of a device for generating an adversarial patch according to an embodiment of the present invention;

[0048] Fig. 9 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0050] Algorithmic adversarial attacks refer to attackers adding subtle perturbations to input data (such as images, videos, and text) to mislead algorithmic models. It can be roughly divided into digital attacks and physical attacks. Digital algorithmic attacks refer to attackers who can directly add subtle perturbations to the input in the digital domain to deceive the model, while physical attacks refer to attackers who do not directly manipulate digital inputs but manipulate target objects in the physical world to indirectly deceive the model. Physical attacks usually need to consider the impact of the actual application environment and overcome environmental interference to ensure the success of the attack. Compared with digital attacks, physical attacks have a more direct impact on real-world applications based on deep learning models, such as autonomous driving and facial recognition payment systems, and increase the security risks of applications.

[0051] As a commonly used physical attack method of intelligent algorithms, adversarial patches can deceive deep learning models by pasting adversarial patches onto targets, such as making the model misclassify the target or making it difficult for the model to detect the target. Although existing adversarial patch attack methods already have good attack capabilities, they ignore the concealment of adversarial patches, making adversarial patches easily detectable by humans and reducing the practicality of the attack. Therefore, developing a high-concealment and fast adversarial camouflage generation technology for the intelligent protection of camouflaged targets in different application environments has become an urgent problem for those skilled in the art to solve.

[0052] Therefore, the embodiments of the present invention provide an adversarial patch generation method, device, computer device, and storage medium to solve the above problems.

[0053] According to an embodiment of the present invention, there is provided an embodiment of an adversarial patch generation method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0054] Figure 1 It is a schematic flowchart of an adversarial patch generation method according to an embodiment of the present invention.

[0055] As Figure 1 shown, an adversarial patch generation method provided in an embodiment of the present invention includes:

[0056] S101: Obtain an initial adversarial patch, an original image, and a background image. Obtain interference information according to a preset method, and add the interference information to the initial adversarial patch to obtain a first adversarial patch.

[0057] The adversarial patch generation method provided in an embodiment of the present invention can be applied to pedestrian target detection.

[0058] In this embodiment, the initial adversarial patch can be set as M∈(0,1) W×H×3 , where W represents the width of the initial adversarial patch, H represents the height of the initial adversarial patch, 3 is the number of image channels, and each channel represents one of the three primary color bases of the tricolor. As an example, it can be initialized to gray, that is, M i,j,k = 0.5, where 0≤i<W, 0≤j<H, k = 0, 1, 2. i represents the index of the image length, j represents the index of the image width, and k represents the index of the RGB three channels of the image.

[0059] The original image is an image marked with a detection frame, which includes the attacked target. When the adversarial patch generation method is applied to pedestrian target detection, the attacked target in the detection frame of the original image is a person, and the background image can be a separately taken environmental image or an environmental image excluding the detection frame in the original image.

[0060] As an example, when the adversarial patch generation method is applied to pedestrian target detection, the INRIA pedestrian dataset can be downloaded and the dataset can be divided into a training set and a test set. The training set is used to train the adversarial patch against the camouflaged base map, and the test set will be used to test the camouflaged attack effect. The single-stage detection model YoloV5 pre-trained on the COCO dataset can be downloaded as the target detection model.

[0061] In this embodiment, in order to simulate the interference occurring in the complex physical world, it is necessary to perform data enhancement on the initial adversarial patch. Interference information can be obtained in a preset manner, and the interference information is added to the initial adversarial patch. After data enhancement on the initial adversarial patch, a first adversarial patch is obtained.

[0062] S102: Add a first adversarial patch to a target area in the original image to obtain a first image with the first adversarial patch.

[0063] In this embodiment, the target area is determined according to the position information of the attacked target in the detection frame of the original image, and the detection frame includes the attacked target. S102 specifically includes: performing a preset transformation process on the first adversarial patch according to the position information of the attacked target in the detection frame of the original image and a preset ratio, adding the first adversarial patch obtained after the preset transformation process to the target area in the original image, and obtaining a first image I with the first adversarial patch M1 M . The preset transformation processing includes but is not limited to rotation transformation, scaling transformation and translation transformation. The preset ratio represents the ratio of the area of ​​the first adversarial patch to the detection frame. The corresponding preset ratio can be determined according to the attacked category, or the preset ratio can be determined according to the ratio of the area of ​​the attacked target to the detection frame, which can improve the adaptability of the adversarial patch. As an example, the ratio of the area of ​​the attacked target to the detection frame can be determined as the preset ratio, and the preset ratio can be 0.25.

[0064] In this embodiment, the first adversarial patch M1 is rotated, scaled and translated using an affine transformation matrix according to the position information of the attacked target in the detection frame of the original image and the preset ratio 0.25 to obtain a first image I with the first adversarial patch M1. M The first image I M It can be determined by the following formula:

[0065] I M=S*R(θ)*M1, where R(θ) represents the rotation matrix of the rotation angle θ, S represents the scaling matrix, only the diagonal is never 0, S x and S y They are the scaling factors corresponding to the x and y axes, respectively, and are defined as follows:

[0066]

[0067] In this embodiment, the affine transformation is an operation performed on the first adversarial patch. The first adversarial patch can be filled with 0 values ​​to the same size as the original image. Then, the rotation information, scaling information, and translation information are determined based on the position information of the attacked target in the detection frame of the original image and the preset ratio. The filled first adversarial patch is rotated, scaled, and translated using radial changes based on the rotation information, scaling information, and translation information. The first adversarial patch is scaled and translated to the position of the attacked target in the original image. Taking pedestrian target detection as an example, the first adversarial patch can be scaled and translated to the position of the person in the original image.

[0068] S103: Determine a target image according to the original image and the first image with the first adversarial patch.

[0069] In this embodiment, the target image is provided with a second adversarial patch determined according to the first adversarial patch.

[0070] In this embodiment, the original image I r and the first image I with the first adversarial patch M Fusion is performed to obtain the target image I with the second adversarial patch A Target image I A It can be determined by the following formula:

[0071]

[0072] Among them, in I M(i,j) =[0,0,0], I A = r(i,j) , in I M(i,j) ≠[0,0,0], I A = M(i,j) .

[0073] S104: Optimize the second adversarial patch through a loss function to obtain a third adversarial patch.

[0074] In this embodiment, S104 optimizes the second adversarial patch M2 through the loss function to obtain the third adversarial patch M3 specifically including:

[0075] S1041: Input the target image into the pre-acquired target detection model to obtain a detection result. The detection result includes the confidence of the detection box in the target image.

[0076] S1042: Determine the loss corresponding to the detection result through a loss function.

[0077] S1043: Optimize the gradient of the second adversarial patch according to the loss corresponding to the detection result to obtain a third adversarial patch.

[0078] In this embodiment, the target image can be input into the pre-acquired target detection model Yolov5, and the detection result output by the target detection model Yolov5 is the confidence F of the detection box in the target image. obj ( A ), determine the maximum value F among the confidences of all detection boxes corresponding to the attacked category v on each target image obj,t ( A ).

[0079] In this embodiment, a loss function needs to be constructed in advance, and the loss function L can be determined by the following formula:

[0080]

[0081] In this embodiment, the loss function can be used to perform gradient backpropagation calculation to optimize the pixel value of each pixel in the second adversarial patch. After multiple rounds of optimization, the optimal third adversarial patch M3 with unlimited color is obtained, such as Figure 2 shown.

[0082] S105: Determine a color replacement matrix of the background image, and determine a color probability matrix of the third adversarial patch.

[0083] In this embodiment, the color replacement matrix is ​​determined based on a preset number of colors that appear most frequently among the colors of all pixels in the background image. The color probability matrix represents the probability of each color in the third adversarial patch.

[0084] In this embodiment, the color that appears more frequently in the third adversarial patch can be replaced by the color that appears more frequently in the background image. The target adversarial patch obtained by color replacement is closer to the color of the background image, which can improve the environmental concealment of the adversarial patch.

[0085] S106: Adjust the color of each pixel in the third adversarial patch according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

[0086] In this embodiment, after obtaining the target adversarial patch, the adversarial patch generation method further includes: testing the attack effect of the target adversarial patch. B Make clothes and wear them on the characters in the original image detection frame to complete the environmental adaptability adversarial camouflage, avoid the detection of the single-stage detection model YoloV5, and achieve deception of the human eye, complete the adversarial camouflage that blends with the background environment, and test the attack effect of the target adversarial patch. The third adversarial patch and the target adversarial patch are compared in terms of adversarial camouflage and environmental concealment. Figure 3 As shown, compared with the third adversarial patch, the target adversarial patch has better adversarial camouflage effect and better environmental concealment.

[0087] The embodiment of the present invention can perform data enhancement on the initial adversarial patch by adding interference information to the initial adversarial patch, thereby increasing the complexity and diversity of the adversarial patch. The first adversarial patch is added to the target area in the original image to obtain a first image with the first adversarial patch, which can simulate the process of adversarial attack. The target image and the second adversarial patch determined according to the first adversarial patch in the target image can be obtained by fusing the original image and the first image with the first adversarial patch. The second adversarial patch is further optimized by the loss function to obtain the third adversarial patch. The color of each pixel in the third adversarial patch is adjusted according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain the target adversarial patch. The main color in the background image is extracted by the color replacement matrix of the background image, and the color of the adversarial patch is replaced, so that adversarial patches in different complex environmental scenes can be quickly generated, and the environmental concealment of the adversarial patch is significantly improved while ensuring the adversarial effect.

[0088] In an optional embodiment, determining the color replacement matrix of the background image in S105 specifically includes: determining the colors of all pixels in the background image. Determining a preset number of colors that appear the most times among the colors of all pixels in the background image. Determining the color replacement matrix of the background image based on the preset number of colors that appear the most times among the colors of all pixels in the background image.

[0089] In this embodiment, the specific value of the preset number is not limited. As an example, when the preset number is 3, for different background environments, the KMEANS clustering algorithm can be used to extract the three primary colors in the background image, that is, the three colors that appear most frequently in the colors of all pixels in the background image. According to the three-primary color light mixing mode, the red, green and blue primary colors of the third adversarial patch can be replaced with the three primary colors extracted from the background image to obtain a target adversarial patch with stronger environmental adaptability. The specific scheme for determining the color replacement matrix of the background image is as follows:

[0090] A real physical environment picture can be taken from the real environment B As a background image, such as Figure 4 As shown. Statistics for each I B The color c of each pixel in i,j As the ambient color set C B =c i,j ∈[0,1] 1×3}, using KMEANS clustering algorithm from C B Extract the three main colors in the environment As shown in the following formula:

[0091]

[0092] Here, 3 means generating three cluster centers. As an example, the three main colors obtained are like Figure 5 As shown. The color replacement matrix C can be generated by converting the three primary colors T :

[0093]

[0094] In an optional embodiment, S106 adjusts the color of each pixel in the third adversarial patch according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch, specifically including: adjusting the color of each pixel in the third adversarial patch according to the product of the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

[0095] In this embodiment, the third adversarial patch M3 is decomposed into a color probability matrix With the red, green and blue primary color matrix C RGB The product of is shown in the following formula:

[0096]

[0097] The red, green and blue primary color matrix C FGB Each row represents a base color, defined as follows:

[0098]

[0099] The color probability matrix of the third adversarial patch Replace the color matrix C with the background image T Multiply to get the target adversarial patch M B , thereby replacing the three primary colors of red, green and blue in the third adversarial patch M3 with the three primary colors extracted from the background environment As the formula shown.

[0100] The target is the adversarial patch M B The value range of is truncated to the range of 0 to 1, as shown in formula M B_C = CLIP(M B ,0,1), where M B_C Represents the color of each pixel in the target adversarial patch. B_C You can get Figure 6 Target adversarial patches shown.

[0101] In an optional implementation, S101 obtains interference information according to a preset method. The present invention does not limit the method of obtaining interference information or the type of interference information. As an example, contrast, brightness and noise can be randomly sampled and generated according to a uniform distribution. In the above example, the interference information includes contrast, brightness and noise.

[0102] In this embodiment, the interference information is used to simulate the color changes in the physical world caused by atmospheric refraction, character movement, camera imaging, etc. Adding the interference information to the initial adversarial patch can adjust the contrast, brightness, noise and other properties of the initial adversarial patch.

[0103] In this embodiment, according to the expected transformation EOT algorithm, physical transformations such as contrast D, brightness B, and noise N in the physical world can be simulated in the digital optimization process of the adversarial patch. The specific scheme of randomly sampling and generating contrast, brightness, and noise in a uniformly distributed manner is as follows:

[0104] In contrast perturbation, a random contrast value D is randomly sampled from a uniform distribution f(x).

[0105] In brightness perturbation, random brightness values ​​B are randomly sampled from a uniform distribution f(x).

[0106] In noise perturbation, a random noise image N is randomly sampled from a uniform distribution f(x).

[0107] Uniform distribution probability density:

[0108]

[0109] Among them, max represents the upper boundary of the uniform distribution interval, max = d + =1.2, min represents the lower boundary of the uniform distribution interval, min=d - =0.8, x represents contrast, brightness or noise.

[0110] The above interference information (contrast D, brightness B and noise N) is added to the initial adversarial patch to obtain the first adversarial patch M1, which is determined by the following formula:

[0111] M1=D*M+B+N The overall process of the anti-patch generation method provided by the present invention is as follows Figure 7 As shown, the technical solution of the present invention can be applied to the complex physical world to effectively reduce the misidentification or missed detection of pedestrians by the target detection system. By extracting the main color from the background image and replacing the base color of the adversarial patch, the environmental concealment of the adversarial patch is significantly improved while ensuring the adversarial effect. By replacing the environmental color, the process of generating and deploying the adversarial patch is accelerated, and the practicality of the adversarial patch in different environmental scenes is improved.

[0112] The combined application of the technical solutions of the present invention can successfully generate environmental color-guided adversarial patches. The adversarial patches generated based on the technical solutions of the present invention can maintain stable offensiveness and strong environmental concealment under various detection models, as well as in the digital world and the physical world. The adversarial patches generated based on the technical solutions of the present invention can well cope with white-box attacks on known model parameters and architectures, mainly attacking the detection confidence score of the single-stage detection model, and can have adversarial attack and environmental concealment effects in both the digital world and the physical world.

[0113] It should be noted that the contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

[0114] In this embodiment, a countermeasure patch generation device is also provided, which is used to implement the above-mentioned embodiment and optional implementation methods, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0115] Figure 8 4 is a structural block diagram of an anti-patch generating device according to an embodiment of the present invention.

[0116] This embodiment provides a device for generating a countermeasure patch, such as Figure 8 As shown, the adversarial patch generating device includes:

[0117] The first processing module 11 is used to obtain an initial adversarial patch, an original image and a background image, obtain interference information according to a preset method, and add the interference information to the initial adversarial patch to obtain a first adversarial patch.

[0118] The second processing module 12 is used to add the first adversarial patch to the target area in the original image to obtain a first image with the first adversarial patch. The target area is determined according to the position information of the attacked target in the detection frame of the original image, and the attacked target is included in the detection frame.

[0119] The third processing module 13 is used to determine a target image according to the original image and the first image with the first adversarial patch. The target image has a second adversarial patch determined according to the first adversarial patch.

[0120] The fourth processing module 14 is used to optimize the second adversarial patch through a loss function to obtain a third adversarial patch.

[0121] The fifth processing module 15 is used to determine the color replacement matrix of the background image and the color probability matrix of the third adversarial patch. The color replacement matrix is ​​determined based on the preset number of colors that appear the most times among the colors of all pixels in the background image. The color probability matrix represents the probability of each color in the third adversarial patch.

[0122] The sixth processing module 16 is used to adjust the color of each pixel in the third adversarial patch according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

[0123] In an optional implementation, the fifth processing module 15 includes:

[0124] The first processing unit is used to determine the colors of all pixels in the background image.

[0125] The second processing unit is used to determine a preset number of colors that appear the most times among the colors of all the pixels in the background image.

[0126] The third processing unit is used to determine a color replacement matrix of the background image according to a preset number of colors that appear most frequently among the colors of all pixels in the background image.

[0127] In an optional implementation, the sixth processing module 16 is specifically configured to adjust the color of each pixel in the third adversarial patch according to the product of the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

[0128] In an optional embodiment, the second processing module 12 is specifically used to perform a preset transformation processing on the first adversarial patch according to the position information of the attacked target in the detection frame of the original image and a preset ratio, and add the first adversarial patch obtained after the preset transformation processing to the target area in the original image to obtain a first image with the first adversarial patch.

[0129] In an optional embodiment, the fourth processing module 14 includes:

[0130] The first processing unit is used to input the target image into a pre-acquired target detection model to obtain a detection result. The detection result includes the confidence of the detection box in the target image.

[0131] The second processing unit is used to determine the loss corresponding to the detection result through a loss function.

[0132] The third processing unit is used to optimize the gradient of the second adversarial patch according to the loss corresponding to the detection result to obtain a third adversarial patch.

[0133] In an optional implementation, the first processing module 11 is specifically configured to generate contrast, brightness and noise by random sampling in a uniformly distributed manner.

[0134] In an optional implementation, the adversarial patch generating device further includes a testing module.

[0135] The testing module is used to test the attack effect of the target adversarial patch after obtaining the target adversarial patch.

[0136] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0137] The anti-patch generation device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0138] The embodiment of the present invention also provides a computer device having the above Figure 8 The adversarial patch generation device shown.

[0139] See also Fig. 9 , Fig. 9 Schematic diagram of the hardware structure of the computer device according to the embodiment of the present invention. Fig. 9 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In an optional embodiment, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0140] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0141] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0142] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function. The data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In an optional embodiment, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0143] The memory 20 may include a volatile memory, such as a random access memory. The memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive. The memory 20 may also include a combination of the above-mentioned types of memory.

[0144] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0145] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc. Further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0146] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0147] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for generating adversarial patches, It is characterized in that include: Get the initial adversarial patch, the original image, and the background image; Acquire interference information according to a preset method, and add the interference information to the initial adversarial patch to obtain a first adversarial patch; Adding the first adversarial patch to a target area in the original image to obtain a first image with the first adversarial patch; The target area is determined according to position information of the attacked target in the detection frame of the original image, and the attacked target is included in the detection frame; Determine a target image according to the original image and the first image with the first adversarial patch; the target image has a second adversarial patch determined according to the first adversarial patch; Optimizing the second adversarial patch by using a loss function to obtain a third adversarial patch; Determine a color replacement matrix of the background image, and determine a color probability matrix of the third adversarial patch; the color replacement matrix is ​​determined based on a preset number of colors that appear the most times among the colors of all pixels in the background image; the color probability matrix represents the probability of each color in the third adversarial patch; The color of each pixel in the third adversarial patch is adjusted according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

2. The method according to claim 1, It is characterized in that Determining the color replacement matrix of the background image includes: Determine the colors of all pixels in the background image; Determine a preset number of colors that appear the most times among the colors of all pixels in the background image; The color replacement matrix of the background image is determined according to a preset number of colors that appear most frequently among the colors of all pixels in the background image.

3. The method according to claim 1 or 2, It is characterized in that The step of adjusting the color of each pixel in the third adversarial patch according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch includes: The color of each pixel in the third adversarial patch is adjusted according to the product of the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

4. The method according to claim 1, It is characterized in that The step of adding the first adversarial patch to the target area in the original image to obtain a first image with the first adversarial patch includes: The first adversarial patch is subjected to a preset transformation process according to the position information of the attacked target in the detection frame of the original image and a preset ratio, and the first adversarial patch obtained after the preset transformation process is added to the target area in the original image to obtain a first image with the first adversarial patch.

5. The method according to claim 1, It is characterized in that The step of optimizing the second adversarial patch by using a loss function to obtain a third adversarial patch includes: Inputting the target image into a pre-acquired target detection model to obtain a detection result; the detection result includes the confidence of the detection box in the target image; Determine the loss corresponding to the detection result through a loss function; The gradient of the second adversarial patch is optimized according to the loss corresponding to the detection result to obtain a third adversarial patch.

6. The method according to claim 1, It is characterized in that The obtaining of interference information according to a preset method includes: generating contrast, brightness and noise by random sampling according to a uniform distribution method.

7. The method according to claim 1, It is characterized in that After obtaining the target antagonistic patch, the method further includes: testing the attack effect of the target antagonistic patch.

8. A device for generating an adversarial patch, It is characterized in that include: A first processing module, for obtaining an initial adversarial patch, an original image and a background image; Acquire interference information according to a preset method, and add the interference information to the initial adversarial patch to obtain a first adversarial patch; A second processing module, configured to add the first adversarial patch to a target area in the original image to obtain a first image with the first adversarial patch; The target area is determined according to position information of the attacked target in the detection frame of the original image, and the attacked target is included in the detection frame; A third processing module, configured to determine a target image according to the original image and the first image with the first adversarial patch; the target image has a second adversarial patch determined according to the first adversarial patch; A fourth processing module, configured to optimize the second adversarial patch by using a loss function to obtain a third adversarial patch; a fifth processing module, configured to determine a color replacement matrix of the background image and a color probability matrix of the third adversarial patch; the color replacement matrix is ​​determined based on a preset number of colors that appear the most frequently among the colors of all pixels in the background image; the color probability matrix represents the probability of each color in the third adversarial patch; The sixth processing module is used to adjust the color of each pixel in the third adversarial patch according to the color replacement matrix of the background image and the color probability matrix of the third adversarial patch to obtain a target adversarial patch.

9. A computer device, It is characterized in that include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the steps of the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the steps of the method according to any one of claims 1 to 7.