Cross-camera physical countermeasure attack method and system for pedestrian detection

Abstract

The invention discloses a cross-camera physical countermeasure attack method and system for pedestrian detection, and relates to the field of artificial intelligence security. The method comprises the following steps: forming a data pair from an original image, a target image obtained by processing the original image through a camera ISP, and randomized parameters obtained by randomizing key parameters; constructing a camera ISP proxy network, and training the camera ISP proxy network according to the data pair to form a camera ISP proxy network model; processing an adversarial patch to be optimized through the camera ISP proxy network model to obtain an ISP processing adversarial patch and applying the ISP processing adversarial patch to a pedestrian data set; and constructing an adversarial optimization framework according to the target detector and the camera ISP proxy network model, and generating a cross-camera adversarial patch in the adversarial optimization framework. The cross-camera adversarial patch obtained by the invention can continuously and effectively attack a target on different pedestrian detection devices, and the success rate of physical adversarial attacks is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security, and in particular to a cross-camera physical adversarial attack method and system for pedestrian detection. Background Art

[0002] In today's digital age, with the growing demand for urban safety, the number of surveillance cameras around the world has exploded. One of the core functions of these surveillance systems is pedestrian detection technology, which uses deep neural networks (DNNs) to analyze surveillance images to accurately identify and track pedestrians. This technology has a wide range of applications and plays an important role in improving public safety, traffic management, and human-computer interaction.

[0003] The application scenarios of pedestrian detection technology are generally:

[0004] (1) For malicious pedestrian detection (such as illegal cameras, armed personnel using drones to identify pedestrians and attack them, etc.), it is necessary to develop a set of attack methods that allow pedestrians to pass through the pedestrian detection model (i.e., reduce the accuracy of pedestrian detection) based on the pedestrian detection technology.

[0005] (2) For normal pedestrian detection (such as identification by legitimate cameras), it is necessary to develop attack methods against pedestrian detection internally and continuously improve the pedestrian detection method based on this method to improve detection accuracy and avoid identity impersonation or evasion of surveillance.

[0006] At present, research shows that the DNN-based pedestrian detection model is relatively vulnerable to physical adversarial attacks. Physical adversarial attacks can be understood as: the attacker misleads the pedestrian detection model by holding an adversarial patch (that is, a carefully designed physical object, such as glasses, clothing or other accessories with special patterns, which can be worn on the body or held in the hand). Summary of the invention

[0007] In view of the defects existing in the prior art, the technical problem solved by the present invention is: for pedestrian detection, how to provide a success rate for physical counterattacks; and thus increase the privacy protection in malicious pedestrian detection scenarios; in normal pedestrian detection scenarios, the pedestrian detection mechanism is improved according to the present invention.

[0008] To achieve the above objectives, in a first aspect, an embodiment of the present application provides a cross-camera physical adversarial attack method for pedestrian detection, the method comprising the following steps:

[0009] Obtain key parameters that affect the image quality based on the camera ISP;

[0010] The original image, the target image obtained by processing the original image through the camera ISP, and the randomized parameters obtained by randomizing the key parameters are formed into a data pair;

[0011] Build a camera ISP proxy network, train the camera ISP proxy network according to the data, evaluate the network training effect by monitoring the error drop curve during the training process, determine the optimal number of training iterations based on the network training effect, and form a camera ISP proxy network model;

[0012] After processing the randomized parameters and the adversarial patch to be optimized through the camera ISP proxy network model, the ISP processed adversarial patch is obtained and applied to the pedestrian dataset, and the pedestrian dataset is input into the target detector;

[0013] By combining the target detector with the camera ISP proxy network model, an adversarial optimization framework is constructed. In the adversarial optimization framework, cross-camera adversarial patches are generated by alternately optimizing the ISP processing adversarial patches and randomizing the parameters.

[0014] In combination with the first aspect, in one embodiment, the process of training the camera ISP proxy network based on the data includes: when downsampling, splicing the randomized parameters with the original image, and downsampling the randomized parameters using an average pooling layer; when upsampling, restoring the size of the randomized parameters to the original size through a deconvolution operation.

[0015] In combination with the first aspect, in one implementation, the key parameters include: a brightness contrast parameter a, a hue saturation parameter b, a gamma parameter γ, a color correction parameter c, a spatial filtering parameter d and a non-local mean e.

[0016] In combination with the first aspect, in one embodiment, it is characterized in that:

[0017] The network structure of the camera ISP proxy network includes:

[0018] L1 = DoubleConv((I;<a,b,γ,c,d,e> ));

[0019] L2 = DoubleConv(Maxpool(L1);<a,b,γ,c,d,e> ));

[0020] L3=DoubleConv(Maxpool(L2;Avgpool(<a,b,γ,c,d,e> )));

[0021] L4=

[0022] DoubleConv(Maxpool(L3;Avgpool(Avgpool(<a,b,γ,c,d,e> ))));

[0023] L5=DoubleConv(Maxpool(L4;Avgpool(Avgpool(Avgpool(<

[0024] a,b,γ,c,d,e>)))));

[0025] L6=DoubleConv(Convtrans(L5));

[0026] L7=DoubleConv(Convtrans(L6));

[0027] L8=DoubleConv(Convtrans(L7));

[0028] L9=DoubleConv(Convtrans(L8));

[0029] L 10 =DoubleConv(Convtrans(L9));

[0030] Among them, I represents the original image; DoubleConv(x) contains two ReLU(BN(Conv(x))) operations; Conv(x) represents the convolution operation on the input x to extract features; BN(x) represents the normalization operation on the input data; ReLU(x) = max(0, x); Maxpool(x) represents the maximum pooling operation on the input data; Avgpool(x) represents the average pooling operation on the input data; Convtrans(x) represents the deconvolution of the input data.

[0031] In combination with the first aspect, in one implementation, the adversarial optimization framework is constructed by combining the target detector with the camera ISP proxy network model. In the adversarial optimization framework, the process of generating cross-camera adversarial patches by alternately optimizing and training the ISP processing adversarial patches and randomized parameters includes:

[0032] The target detector calculates the total loss through the overall loss function and designs the target optimization function based on the overall loss function;

[0033] The camera ISP proxy network model calculates the loss through the camera ISP loss function, and designs the target optimization function according to the camera ISP loss function;

[0034] During the alternating optimization training process, we alternate between optimizing ISP to handle adversarial patches and optimizing randomization parameters.

[0035] In combination with the first aspect, in one implementation, the overall loss function Including the target loss function of the target detector And the total variation loss function

[0036]

[0037] Where λ1 represents the target loss weight coefficient, and λ2 represents the total variation loss weight coefficient; specifically, λ1=1, λ2=0.5 can be set to balance and

[0038]

[0039] in, Refers to the confidence of the pedestrian class in the prediction results of the target detector, I patch Refers to the image with the ISP processed adversarial patch applied;

[0040]

[0041] Among them, p i,j represents the pixel value at coordinate (i, j) in the adversarial patch p.

[0042] In combination with the first aspect, in one implementation, the camera ISP loss function The calculation formula is:

[0043]

[0044] In combination with the first aspect, in one implementation, the calculation formula of the objective optimization function is:

[0045]

[0046] Where P represents the cross-camera adversarial patch and f ISP Represents the ISP proxy network model, I SCENE represents the physical scene, f represents the target detector, and GT represents the actual label.

[0047] In combination with the first aspect, in one implementation, the acquisition conditions of the original image include: belonging to a pedestrian category and meeting specified image parameters; the determination process of the original image includes: screening images corresponding to the processing conditions in a data set in RGB format;

[0048] The process of performing random processing on the key parameters to obtain randomized parameters includes: performing random processing on the key parameters within their value ranges using a camera ISP simulation tool to obtain randomized parameters;

[0049] The camera ISP proxy network is a differentiable camera ISP proxy network based on Unet; the convolution kernel size used by the camera ISP proxy network is 3×3, the stride of the pooling layer is 2×2, the number of feature channels of the encoder is {32, 64, 128, 256, 512}, and the number of feature channels of the decoder is {32, 64, 128, 256, 512}.

[0050] In a second aspect, an embodiment of the present application provides a cross-camera physical adversarial attack system for pedestrian detection, which is used to implement the method provided in the first aspect.

[0051] Compared with the prior art, the advantages of the present invention are:

[0052] The present invention trains a camera ISP proxy network to obtain a camera ISP proxy network model by including data pairs formed by key parameters that have a significant impact on imaging quality, original images, and target images, thereby improving the accuracy of physical adversarial attacks.

[0053] The present invention uses the object detector of the pedestrian data set with ISP processing adversarial patches as the attack module, and the camera ISP proxy network model as the defense module to build an adversarial optimization framework, which gives the attack patch the ability to attack across cameras. By optimizing the ISP parameters to improve the defense capability of the camera ISP module and minimize the attack effect of the adversarial patch, a zero-sum game is formed with the adversarial patch optimization module, thereby enhancing the attack capability of the adversarial patch.

[0054] Through real-world experiments conducted on a variety of imaging devices, including two different camera models (Sony and Canon) and four smartphone cameras (iPhone, Redmi, Huawei and Samsung), it is demonstrated that the above scheme can consistently and effectively achieve the attack goal, that is, improve the success rate of physical adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0056] Figure 1Schematic diagram of comparison of attack effects of different adversarial patches under 50 randomly selected camera ISPs in an embodiment of the present invention;

[0057] Figure 2 Schematic diagram of the process of a cross-camera physical adversarial attack method for pedestrian detection in an embodiment of the present invention;

[0058] Figure 3 Schematic diagram of the attack effects of six different adversarial patches on six different cameras in an embodiment of the present invention;

[0059] Figure 4 Schematic diagram of attack effect comparison between AdvPatch and T-SEA in an embodiment of the present invention;

[0060] Figure 5 Schematic diagram of the architecture of a cross-camera physical adversarial attack system for pedestrian detection in an embodiment of the present invention;

[0061] Figure 6 This is a schematic diagram of the hardware structure of a cross-camera physical counter-attack device for pedestrian detection involved in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0063] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0064] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0065] First, the research and development process of the present invention is briefly described.

[0066] The applicant has found through research and development that existing physical adversarial attacks ignore the impact of the camera image processing system (Image Signal Processing, ISP) on the attack effect. The camera ISP is the software inside the camera that is responsible for processing the raw data captured from the image sensor and converting it into images that can be analyzed. Different camera ISP processing algorithms and parameter settings may cause the performance of the adversarial patch in the digital space to differ significantly from its actual effect in the physical world.

[0067] For this purpose, see Figure 1 As shown, the applicant analyzed the attack effect of the adversarial patch (AP value in the digital space) obtained in the digital space in the scenarios of adding and not adding the camera ISP. Figure 1 The vertical axis is the pedestrian detection accuracy (AP), and the horizontal axis is different adversarial patches. The light gray represents the pedestrian detection accuracy in the scene without camera ISP (evaluation w / o ISP), and the dark gray represents the pedestrian detection accuracy in the scene with 50 random camera ISPs (evaluation w / o 50ISPs). It can be concluded that the attack effects of different adversarial patches on different camera ISPs are significantly different, and the detection accuracy of most adversarial patches in the scene with camera ISP is higher than that in the scene without ISP.

[0068] This means that a well-designed adversarial patch in the digital space may fail under the real-world camera ISP, thus reducing the success rate of the attack.

[0069] On this basis, first, see Figure 2 As shown, the embodiment of the present application provides a cross-camera physical adversarial attack method for pedestrian detection, the steps of the method include:

[0070] S1: Select key parameters that have a significant impact on the image quality from the color tone module and denoising module of the camera ISP.

[0071] S2: Processing the original image that meets the acquisition conditions through the camera ISP to obtain a target image; randomizing the key parameters of the original image to obtain randomized parameters; forming a data pair with the original image, the target image of the original image, and the randomized parameters of the original image.

[0072] In one embodiment, the acquisition conditions in S2 include: belonging to the pedestrian category and specified image parameters (the bounding box height multiplied by the image height is greater than 140 pixels); the process of determining the original image in S2 includes: screening images corresponding to the acquisition conditions in a data set in RGB format (COCO data set); on this basis, this embodiment subsequently obtained 2270 data pairs.

[0073] At the same time, the process of randomizing the key parameters of the original image in S2 to obtain randomized parameters includes: using the camera ISP simulation tool (Fast-OpenISP open source library) to randomize the key parameters within their value range to obtain randomized parameters (if there are 6 key parameters above, six-dimensional randomized parameters will be obtained).

[0074] S3: Construct a camera ISP proxy network, and train the camera ISP proxy network according to the data pairs (i.e., input the data pairs into the camera ISP proxy network for network training and optimization of randomization parameters, use the images processed by ISP as the target images during training, use 90% of the sample set for training, and use the remaining data for testing), and evaluate the network training effect by monitoring the error reduction curve during the training process (in this example, ordinary technicians in the field can obtain specific training and evaluation methods based on the error reduction curve, which is an evaluation condition for the network training effect), and form a camera ISP proxy network model after determining the optimal number of training iterations based on the network training effect.

[0075] S4: Initialize a random noise as the adversarial patch to be optimized, and process the adversarial patch to be optimized and the above-mentioned randomized parameters through the camera ISP proxy network model to obtain the ISP-processed adversarial patch; under the condition of considering the target position information, the ISP-processed adversarial patch is applied to the pedestrian dataset (INRIAPerson).

[0076] S5: The pedestrian dataset with ISP-processed adversarial patches in S4 is input into the target detector (YOLOv5, You Only Look Once, target detection algorithm version 5 in this embodiment) as an attack module, and the camera ISP proxy network model in S3 is used as a defense module; an adversarial optimization framework is constructed by combining the attack module with the defense module, and a cross-camera adversarial patch is generated in the adversarial optimization framework by alternately optimizing the ISP-processed adversarial patch and randomized parameters until the total loss function converges.

[0077] In one embodiment, the camera ISP proxy network is a differentiable camera ISP proxy network based on Unet (a convolutional neural network for image segmentation); after the ISP proxy network model is obtained, it can be used for subsequent simulation of diverse camera ISP configurations.

[0078] In one embodiment, the convolution kernel size used by the camera ISP proxy network is 3×3, the stride of the pooling layer is 2×2, the number of feature channels of the encoder is {32, 64, 128, 256, 512}, and the number of feature channels of the decoder is {32, 64, 128, 256, 512}.

[0079] In one embodiment, the key parameters in S1 include:

[0080] The brightness contrast parameter a is obtained from the brightness contrast control module of the color and hue correction module, and the specific parameter is 64 to 256;

[0081] The hue saturation parameter b is obtained from the hue saturation control module of the color and tone correction module, and the specific parameter is 64 to 256;

[0082] The gamma parameter γ is obtained from the gamma adjustment control module of the color and tone correction module, and the specific parameter is 0.4 to 2.0;

[0083] The color correction parameter c is obtained from the color correction matrix of the color and hue correction module, and the specific parameter is 512 to 1024;

[0084] The spatial filter parameter d is obtained from the spatial filter module of the denoising module, and the specific parameter is 0.1 to 2.0;

[0085] The non-local mean e is obtained from the non-local mean filter module of the denoising module, and the specific parameter is 1.0 to 32.0.

[0086] On this basis, the process of training the camera ISP proxy network according to the data pair in S3 includes: in the forward propagation process of the camera ISP proxy network, the randomized parameters of the data pair are used as input, standardized to the interval [0,1], and spliced ​​with the original image of the data pair and then input into the camera ISP proxy network;

[0087] In each downsampling step of the input process, the randomized parameters are concatenated with the original image to preserve the parameter information; at the same time, the randomized parameters are downsampled using an average pooling layer to match the size of the original image;

[0088] At each upsampling step of the input process, the size of the randomized parameters is restored to the original size through a deconvolution operation so as to be concatenated with the feature map.

[0089] Specifically, the network structure of the camera ISP proxy network is as follows:

[0090] L1 = DoubleConv((I;<a,b,γ,c,d,e> ));

[0091] L2 = DoubleConv(Maxpool(L1);<a,b,γ,c,d,e> ));

[0092] L3=DoubleConv(Maxpool(L2;Avgpool(<a,b,γ,c,d,e> )));

[0093] L4=

[0094] DoubleConv(Maxpool(L3;Avgpool(Avgpool(<a,b,γ,c,d,e> ))));

[0095] L5=DoubleConv(Maxpool(L4;Avgpool(Avgpool(Avgpool(<

[0096] a,b,γ,c,d,e>)))));

[0097] L6=DoubleConv(Convtrans(L5));

[0098] L7=DoubleConv(Convtrans(L6));

[0099] L8=DoubleConv(Convtrans(L7));

[0100] L9=DoubleConv(Convtrans(L8));

[0101] L 10 =DoubleConv(Convtrans(L9));

[0102] Among them, I represents the original image; DoubleConv(x) contains two ReLU9BN(Conv(x)) operations; Conv(x) represents the convolution operation on the input x to extract features; BN(x) represents the normalization operation on the input data; ReLU(x) = max(0, x); Maxpool(x) represents the maximum pooling operation on the input data; Avgpool(x) represents the average pooling operation on the input data; Convtrans(x) represents the deconvolution of the input data.

[0103] In one embodiment, the process of processing the adversarial patch to be optimized and the above-mentioned randomized parameters through the camera ISP proxy network model in S4 includes: performing multi-step processing on the anti-patch to be optimized, including rotating, adding noise, enlarging or reducing the perturbed image to generate an adversarial sample with high attack performance.

[0104] In one embodiment, the process of S5 specifically includes:

[0105] The attack module (target detector) calculates the total loss through the overall loss function, designs the target optimization function according to the overall loss function, and uses the Adam optimizer to update the adversarial patch to maximize its attack performance;

[0106] The defense module (camera ISP proxy network model) calculates the loss through the camera ISP loss function, designs the target optimization function according to the camera ISP loss function, and uses the Adam optimizer to update the randomization parameters to minimize the attack performance of the adversarial patch;

[0107] During the alternating optimization training process, we alternate between optimizing the ISP to process adversarial patches and optimizing the randomization parameters. After experimental verification, setting the number of alternations to 20 (optimizing the ISP to process adversarial patches 20 times and optimizing the randomization parameters 20 times) can ensure that both the attack module and the defense module have reached the optimal optimization state.

[0108] Furthermore, the above overall loss function Including the target loss function of the target detector And the total variation loss function

[0109] Where λ1 represents the target loss weight coefficient, and λ2 represents the total variation loss weight coefficient; specifically, λ1=1, λ2=0.5 can be set to balance and

[0110]

[0111] in, Refers to the confidence of the pedestrian class in the prediction results of the target detector, I patch Refers to the image with the ISP processed adversarial patch applied;

[0112]

[0113] Among them, p i,j represents the pixel value at coordinate (i, j) in the adversarial patch p.

[0114] On this basis, the above camera ISP loss function The calculation formula is:

[0115]

[0116] At the same time, the calculation formula of the above objective optimization function is:

[0117]

[0118] Where P represents the cross-camera adversarial patch and f ISP Represents the ISP proxy network model, I SCENErepresents the physical scene, f represents the target detector, and GT represents the actual label; that is, maximize the gap between the detector output prediction and the actual label to generate the most aggressive cross-camera adversarial patch to the target detector.

[0119] The following compares the attack effects of different adversarial patches on different cameras.

[0120] See also Figure 3 As shown, Figure 3 The middle horizontal axis is two different camera models (Sony and Canon) and four smartphone cameras (iPhone, Redmi, Huawei and Samsung), and the horizontal axis is the attack success rate (ASR). Figure 6 Ours is the cross-camera adversarial patch of this application, and the rest are other adversarial patches. It can be seen that the cross-camera adversarial patch of this application has good generalization and can implement effective physical adversarial attacks (with high attack success rates) for pedestrian detection under various camera conditions.

[0121] In the meantime, see Figure 4 As shown in Figure 2, compared with AdvPatch and T-SEA, the cross-camera adversarial patch of this application can continuously and effectively attack the target detector on a variety of imaging devices including the above two different models of cameras and four smartphone cameras. Figure 4 The presence of a border enclosing a person indicates that it has been detected).

[0122] In a second aspect, an embodiment of the present application further provides a cross-camera physical adversarial attack system for pedestrian detection, which is used to implement the steps of the method mentioned in the first aspect.

[0123] See also Figure 5 As shown, the system specifically includes a key parameter selection module, a data pair construction module, a camera ISP proxy network model training module, an adversarial patch optimization module, and a cross-camera adversarial patch generation module.

[0124] The key parameter selection module is used for: the process of S1 in the above method.

[0125] The data pair building module is used for: the process of S2 in the above method.

[0126] The camera ISP proxy network model training module is used for: the process of S3 in the above method.

[0127] The adversarial patch optimization module is used for: the process of S4 in the above method.

[0128] The cross-camera adversarial patch generation module is used for: the process of S5 in the above method.

[0129] Among them, the functional implementation of each module in the above-mentioned cross-camera physical counter-attack device for pedestrian detection corresponds to the steps in the above-mentioned cross-camera physical counter-attack method embodiment for pedestrian detection, and their functions and implementation processes will not be repeated here one by one.

[0130] On the third aspect, an embodiment of the present application provides a cross-camera physical counter-attack device for pedestrian detection. The cross-camera physical counter-attack device for pedestrian detection can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0131] Reference Figure 6 , Figure 6 The hardware structure diagram of the cross-camera physical counter-attack device for pedestrian detection involved in the embodiment of the present application is shown in FIG. In the embodiment of the present application, the cross-camera physical counter-attack device for pedestrian detection may include a processor, a memory, a communication interface, and a communication bus.

[0132] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0133] The communication interface includes an input / output (I / O) interface, a physical interface, and a logical interface, which are used to interconnect the devices inside the cross-camera physical counter-attack device for pedestrian detection, and an interface for interconnecting the cross-camera physical counter-attack device for pedestrian detection with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.

[0134] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0135] The processor may be a general-purpose processor, which may call a cross-camera physical adversarial attack program for pedestrian detection stored in a memory, and execute the cross-camera physical adversarial attack method for pedestrian detection provided in an embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the cross-camera physical adversarial attack program for pedestrian detection is called may refer to the various embodiments of the cross-camera physical adversarial attack method for pedestrian detection in the present application, which will not be repeated here.

[0136] Those skilled in the art will understand that Figure 6 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0137] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0138] The computer-readable storage medium of the present application stores a cross-camera physical adversarial attack program for pedestrian detection, wherein when the cross-camera physical adversarial attack program for pedestrian detection is executed by a processor, the steps of the cross-camera physical adversarial attack method for pedestrian detection as described above are implemented.

[0139] Among them, the method implemented when the cross-camera physical adversarial attack program for pedestrian detection is executed can refer to the various embodiments of the cross-camera physical adversarial attack method for pedestrian detection in this application, and will not be repeated here.

[0140] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0141] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0142] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0143] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0144] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0145] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0147] The above are only specific implementations of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the embodiments of the present invention, and these modifications or replacements should be included in the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention shall be based on the protection scope of the claims.

Claims

1. A cross-camera physical adversarial attack method for pedestrian detection, characterized in that: The method comprises the following steps: Obtain key parameters that affect the image quality based on the camera ISP; The original image, the target image obtained by processing the original image through the camera ISP, and the randomized parameters obtained by randomizing the key parameters are formed into a data pair; Build a camera ISP proxy network, train the camera ISP proxy network according to the data, evaluate the network training effect by monitoring the error drop curve during the training process, determine the optimal number of training iterations based on the network training effect, and form a camera ISP proxy network model; After processing the randomized parameters and the adversarial patch to be optimized through the camera ISP proxy network model, the ISP processed adversarial patch is obtained and applied to the pedestrian dataset, and the pedestrian dataset is input into the target detector; By combining the target detector with the camera ISP proxy network model, an adversarial optimization framework is constructed. In the adversarial optimization framework, cross-camera adversarial patches are generated by alternately optimizing the ISP processing adversarial patches and randomizing the parameters.

2. The cross-camera physical adversarial attack method for pedestrian detection according to claim 1, characterized in that: The process of training the camera ISP proxy network based on the data includes: when downsampling, splicing the randomized parameters with the original image, and downsampling the randomized parameters using the average pooling layer; when upsampling, restoring the size of the randomized parameters to the original size through a deconvolution operation.

3. The cross-camera physical adversarial attack method for pedestrian detection according to claim 1, characterized in that: The key parameters include: brightness contrast parameter a, hue saturation parameter b, gamma parameter γ, color correction parameter c, spatial filtering parameter d and non-local mean e.

4. The cross-camera physical adversarial attack method for pedestrian detection according to claim 3, characterized in that: The network structure of the camera ISP proxy network includes: L1 = DoubleConv((I;<a,b,γ,c,d,e> )); L2 = DoubleConv(Maxpool(L1);<a,b,γ,c,d,e> )); L3=DoubleConv(Maxpool(L2;Avgpool(<a,b,γ,c,d,e> ))); L4＝ DoubleConv(Maxpool(L3;Avgpool(Avgpool(<a,b,γ,c,d,e> )))); L5=DoubleConv(Maxpool(L4;Avgpool(Avgpool(Avgpool(< a,b,γ,c,d,e>))))); L6=DoubleConv(Convtrans(L5)); L7=DoubleConv(Convtrans(L6)); L8=DoubleConv(Convtrans(L7)); L9=DoubleConv(Convtrans(L8)); L 10 =DoubleConv(Convtrans(L9)); Among them, I represents the original image; DoubleConv(x) contains two ReLU(BN(Conv(x))) operations; Conv(x) represents the convolution operation on the input x to extract features; BN(x) represents the normalization operation on the input data; ReLU(x) = max(0, x); Maxpool(x) represents the maximum pooling operation on the input data; Avgpool(x) represents the average pooling operation on the input data; Convtrans(x) represents the deconvolution of the input data.

5. The cross-camera physical adversarial attack method for pedestrian detection according to claim 1, characterized in that: The adversarial optimization framework is constructed by combining the target detector with the camera ISP proxy network model. In the adversarial optimization framework, the process of generating cross-camera adversarial patches by alternately optimizing the ISP processing adversarial patches and randomizing parameters includes: The target detector calculates the total loss through the overall loss function and designs the target optimization function based on the overall loss function; The camera ISP proxy network model calculates the loss through the camera ISP loss function, and designs the target optimization function according to the camera ISP loss function; During the alternating optimization training process, we alternate between optimizing ISP to handle adversarial patches and optimizing randomization parameters.

6. The cross-camera physical adversarial attack method for pedestrian detection according to claim 5, characterized in that: The overall loss function Including the target loss function of the target detector And the total variation loss function Where λ1 represents the target loss weight coefficient, and λ2 represents the total variation loss weight coefficient; specifically, λ1=1, λ2=0.5 can be set to balance and in, Refers to the confidence of the pedestrian class in the prediction results of the target detector, I patch Refers to the image with the ISP processed adversarial patch applied; Among them, p i,j represents the pixel value at coordinate (i, j) in the adversarial patch p.

7. The cross-camera physical adversarial attack method for pedestrian detection according to claim 5, characterized in that: The camera ISP loss function The calculation formula is:

8. The cross-camera physical adversarial attack method for pedestrian detection as claimed in claim 5, characterized in that: The calculation formula of the objective optimization function is: Where P represents the cross-camera adversarial patch and f ISP Represents the ISP proxy network model, I SCENE represents the physical scene, f represents the target detector, and GT represents the actual label.

9. The cross-camera physical adversarial attack method for pedestrian detection according to any one of claims 1 to 8, characterized in that: The acquisition conditions of the original image include: belonging to the pedestrian category and meeting the specified image parameters; the determination process of the original image includes: screening the image corresponding to the processing conditions in the RGB format data set; The process of performing random processing on the key parameters to obtain randomized parameters includes: performing random processing on the key parameters within their value ranges using a camera ISP simulation tool to obtain randomized parameters; The camera ISP proxy network is a differentiable camera ISP proxy network based on Unet; the convolution kernel size used by the camera ISP proxy network is 3×3, the stride of the pooling layer is 2×2, the number of feature channels of the encoder is {32, 64, 128, 256, 512}, and the number of feature channels of the decoder is {32, 64, 128, 256, 512}.

10. A cross-camera physical adversarial attack system for pedestrian detection, characterized in that: The system is used to implement the method described in any one of claims 1 to 9.

Images