Adversarial attack method and device for crowd counting scenarios

Abstract

The application discloses a kind of crowd counting scene-oriented counterattack method and device.The method comprises the following steps: random scaling countermeasure sample, obtain differentiable interpolation module;Extract the attention representation of crowd counting model for differentiable interpolation module, obtain the heat map of corresponding representation model attention;By differentiable interpolation module and the heat map of corresponding representation model attention, introduce loss function, optimize countermeasure sample, to attack crowd counting model.The application solves the problem of poor cross-model migration ability in traditional counterattack, makes the counterattack effect in crowd counting scene far superior to prior art.

Description

Technical Field

[0001] This invention relates to an adversarial attack method for crowd counting scenarios, and also to a corresponding adversarial attack device, belonging to the field of computer vision detection. Background Technology

[0002] The field of artificial intelligence (AI) has always focused on real-world applications. In recent years, deep learning has achieved remarkable success in several challenging areas such as computer vision and natural language processing, and it has become a core module of AI products. As a subfield, the safety issues inherent in AI and its applications in human safety have also attracted widespread attention. Crowd counting is a typical security application scenario, aiming to count the number of people in unconstrained real-world environments. It is commonly used in security fields such as parade monitoring, public place crowd flow monitoring, and public transportation crowd density monitoring. Currently, engineers are using deep learning technology to develop AI camera systems for crowd counting scenarios, which can replace the human eye in achieving crowd flow statistics and crowd density monitoring, thus safeguarding public safety.

[0003] In recent years, adversarial examples have gradually become a research hotspot in the field of deep learning. Since decision-making in crowd counting scenarios relies on deep learning models, their robustness is also threatened by adversarial examples. Therefore, in scenarios where adversarial examples exist, the security and robustness of crowd counting models face significant vulnerabilities. To address these needs, researchers have proposed many methods for attacking deep learning models. However, these attack methods generally have poor transferability and are often unapplicable in real-world scenarios.

[0004] Chinese invention patent ZL 202110588717.2 discloses a momentum-based method for generating adversarial examples for a crowd counting model. The method includes the following steps: inserting tensors to generate adversarial patches; adding adversarial patches to the original image to generate initial adversarial examples; inputting the adversarial examples into the crowd counting model and outputting a predicted crowd density map; calculating the loss value between the predicted crowd density map and the target crowd density map; optimizing the adversarial examples whose loss value exceeds a threshold using momentum-based adversarial patches, and then executing the step of outputting the predicted crowd density map, until adversarial examples whose loss value does not exceed the threshold are obtained. The adversarial patches are difficult for the human eye to detect in a high-density background; momentum is also integrated into the optimization process of the adversarial patches, overcoming the problem of getting trapped in poor local minima or maxima when searching for the best adversarial example point in high-dimensional space. This method can effectively evaluate the robustness of the crowd counting model against adversarial patch attacks and authentication defenses. Summary of the Invention

[0005] The primary technical problem to be solved by this invention is to provide an adversarial attack method for crowd counting scenarios.

[0006] Another technical problem to be solved by the present invention is to provide an adversarial attack device for crowd counting scenarios.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] According to a first aspect of the present invention, an adversarial attack method for crowd counting scenarios is provided, comprising the following steps:

[0009] By randomly scaling adversarial examples, a differentiable interpolation module is obtained;

[0010] Extract the attention representation of the crowd counting model for the differentiable interpolation module to obtain a heatmap of the corresponding attention representation model;

[0011] By introducing a loss function through a differentiable interpolation module and a heatmap representing the attention of the corresponding model, adversarial examples are optimized to attack the crowd counting model.

[0012] Preferably, before the step of randomly scaling the adversarial examples, the method further includes:

[0013] The adversarial patch is initialized randomly, and the position matrix is ​​generated randomly to obtain the adversarial sample.

[0014] Preferably, the randomly scaled adversarial examples specifically include:

[0015] The size of the adversarial examples is randomly adjusted using the probability of the interpolation operation.

[0016] Preferably, the step of introducing the loss function through the differentiable interpolation module and the heatmap corresponding to the attention of the representation model specifically includes:

[0017] The scale-aware loss is obtained through a differentiable interpolation module;

[0018] The position-aware loss is obtained by using the heatmap corresponding to the attention of the model.

[0019] Preferably, the optimized adversarial examples specifically include:

[0020] Gradient-based iterative algorithms update adversarial examples until the loss function reaches a preset condition.

[0021] Preferably, the expression for the differentiable interpolation module is:

[0022]

[0023] In the above formula, p is the probability of performing interpolation, a and b are the lower and upper limits of the image scaling ratio, respectively, and sampling(·) represents upsampling and downsampling operations.

[0024] Preferably, the expression for the heatmap representing the attention of the corresponding model is:

[0025]

[0026] In the above formula, C is the number of people predicted by the crowd counting model, which is the summation of the output of the crowd counting model pixel by pixel. Let Z represent the pixel value at coordinates (i, j) of the k-th feature map of the model, Z is a hyperparameter used for global average pooling, and ReLU(·) represents the rectified linear function. This represents the density attention module.

[0027] Preferably, the expression for the scale-aware loss is:

[0028]

[0029] In the above formula, f θi,j The pixel value at location coordinates (i, j) of the predicted density map output by the crowd counting model, T(x adv ) is a differentiable interpolation module;

[0030] The expression for the location-aware loss is:

[0031]

[0032] In the above formula, S i,j The pixel value at position coordinates (i, j) of the heatmap representing the model's attention.

[0033] Preferably, the loss function expression is as follows:

[0034]

[0035] In the above formula, For scale perception loss, This is due to the loss of location perception.

[0036] According to a second aspect of the present invention, an adversarial attack device for crowd counting scenarios is provided, comprising a processor and a memory, wherein the processor reads a computer program in the memory for performing the following operations:

[0037] By randomly scaling adversarial examples, a differentiable interpolation module is obtained;

[0038] Extract the attention representation of the crowd counting model for the differentiable interpolation module to obtain a heatmap of the corresponding attention representation model;

[0039] By introducing a loss function through a differentiable interpolation module and a heatmap representing the attention of the corresponding model, adversarial examples are optimized to attack the crowd counting model.

[0040] In this invention, a differentiable interpolation module is used to adapt to different crowd scale information under multiple receptive fields of different crowd counting models, and a density attention mechanism is used to perturb the shared crowd position perception among crowd counting models. This improves the attack invariance of adversarial patches against crowd counting models and enhances the transferability of adversarial examples. This invention solves the problem of poor cross-model transferability in traditional adversarial attacks, making the adversarial attack performance in crowd counting scenarios far superior to existing technologies. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of crowd counting in different scenarios in an embodiment of the present invention;

[0042] Figure 2 This is a flowchart of an adversarial attack method for crowd counting scenarios in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the adversarial example generation process in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of an adversarial attack device for crowd counting scenarios in an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram illustrating the behavior of the generated adversarial patch in the real physical world in an embodiment of the present invention;

[0046] Figure 6 The graph shows a comparison of the model performance of the adversarial attack method provided by this invention in three test sets (Distractors, Weather, and Negative samples).

[0047] Figure 7 This is a comparison chart showing the effectiveness of the crowd counting model trained in this invention in resisting crowd noise compared to the original model. Detailed Implementation

[0048] The technical content of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0049] Crowd counting refers to estimating the number of people in an unrestricted setting. It is used in many critical security scenarios (such as gatherings or pedestrian density monitoring, etc.). Figure 1As shown in (a) and (b), crowd counting is becoming increasingly important. To date, research on crowd counting models has mainly focused on designing different crowd counting methods, including detection-based methods and density map-based convolutional neural network estimation methods. Among these, density map-based convolutional neural network estimation methods have become a common solution for crowd counting due to their better performance. However, density map-based convolutional neural network estimation methods are very vulnerable to adversarial attacks using adversarial examples.

[0050] Adversarial examples are samples generated by making minor adjustments to the original samples. They have no impact on human cognition or object recognition, but they can mislead neural networks into making incorrect decisions, posing a serious security threat to the practical application of machine learning in the digital and physical worlds. Since crowd counting models rely on deep learning models for decision-making, their robustness is also threatened by adversarial examples.

[0051] In one embodiment of the invention, adversarial examples refer to highly sophisticated noise that is indistinguishable to the human eye but devastating to neural networks.

[0052] F θ (x adv )≠y st||xx adv ||<ε

[0053] Where x represents a normal sample, x adv Let x represent an adversarial example, and y represent the class. adv They are visually similar and the distance between them is less than ε, but the neural network F will misclassify them.

[0054] Obtain from the monitor, such as Figure 1 As shown, (a) illustrates a digital environment attack. The scenario involves an electronic billboard in a subway passageway with an actual population of 133. The left image of (a) shows a clean scenario, while the right image shows the scenario with an adversarial attack. In the right image of (a), an adversarial patch is applied to the electronic billboard. The crowd counting model estimates 140 people for the left image of (a) and 936 people for the right image of (a).

[0055] Figure 1 (b) shows a physical environment attack. In the actual scenario of a subway station, the actual number of people is 37. The left image of (b) shows a clean scene, and the crowd counting model estimates the number of people to be 38. The right image of (b) shows an adversarial patch pasted on the wall, and the crowd counting model estimates the number of people to be 3881.

[0056] While adversarial attacks are detrimental to crowd counting models (i.e., neural network models), they are valuable for gaining insights into the blind spots of neural networks and can help improve their robustness.

[0057] like Figure 1 (c) shows the performance of the crowd counting model after training with adversarial examples in a complex background. As shown in (c), the actual number of people is 12 in a smoggy environment. The left side of (c) shows that the crowd counting model estimates 1 person. The right side of (c) shows that the crowd counting model estimates 8 people after adversarial training.

[0058] Therefore, although adversarial attacks can harm crowd counting models, adversarial training can improve the robustness of crowd counting models.

[0059] On the other hand, the unique characteristic of crowd counting tasks lies in the fact that different crowd counting models possess different receptive fields, resulting in varying preferences for perceiving information at different crowd scales—that is, diverse crowd scale perceptions. Attacking only the model's perception at a specific scale often leads to overfitting, thereby weakening the cross-model transferability of the attack sample. Furthermore, different crowd counting models share a common crowd location perception; that is, different models rely on similar density attention representations for final predictions, and in crowd counting scenarios, this attention is often dispersed across the location of the crowd. Perturbating this shared density attention representation using adversarial examples can significantly improve the transferability of adversarial examples across different crowd counting models.

[0060] like Figure 2 As shown, the adversarial attack method for crowd counting scenarios provided in this embodiment of the invention includes at least the following steps:

[0061] 101. Randomly scale the adversarial examples to obtain a differentiable interpolation module;

[0062] 102. Extract the attention representation of the crowd counting model for the differentiable interpolation module, and obtain the heatmap of the corresponding representation model attention;

[0063] 103. By introducing a loss function through a differentiable interpolation module and a heatmap representing the attention of the corresponding model, adversarial examples are optimized to attack the crowd counting model.

[0064] Before step 101 of randomly scaling the adversarial example, the following steps are also included:

[0065] 100. Randomly initialize adversarial patches and randomly generate position matrices to obtain adversarial samples.

[0066] For a crowd counting model, given the original input image x, design the model f.θ An approximate solution to the optimization problem of density map I in the real world:

[0067]

[0068] In the formula, N is the number of input samples.

[0069] In embodiments of the present invention, such as Figure 3 As shown, the goal is to generate an adversarial patch δ to deceive the model f. θ Making incorrect predictions. Specifically, given a population counting model f θ By maximizing the model loss, an adversarial patch δ is also generated:

[0070]

[0071] In the formula, the adversarial sample x adv Composed of a normal sample (clean sample) x, an adversarial patch δ, and a position matrix M, it can be represented as:

[0072] x adv =(1-M)⊙x+M⊙δ (I)

[0073] In formula (1), is the ordinary sample, x is the original image, δ is the adversarial patch, M is the position matrix, and ⊙ represents element multiplication.

[0074] Therefore, the generated adversarial patch can mislead the crowd counting model, leading to incorrect predictions.

[0075] The invariant properties of crowd counting models significantly impact the transferability of adversarial attacks. Therefore, it is necessary to identify model-shared features that highly influence model performance, and then learn the model's invariant features from them to generate adversarial patches that can be transferred between crowd counting models.

[0076] like Figure 3 As shown, in step 101, randomly scaling the adversarial examples specifically includes:

[0077] 1011. The size of the adversarial sample is randomly adjusted with the probability of the interpolation operation.

[0078] In this embodiment of the invention, differentiable interpolation is used to randomly scale adversarial examples. The expression for the differentiable interpolation module is:

[0079]

[0080] In formula (2), p is the probability of performing interpolation, a and b are the lower and upper limits of the image scaling ratio, sampling(·) represents upsampling and downsampling operations, and T represents a differentiable interpolation module.

[0081] Crowd scale variations significantly impact the design of estimation-based crowd counting methods. Different crowd counting models contain different receptive fields and exhibit varying perceptual preferences for crowds of different sizes. Capturing scale-invariant features between crowd counting models helps adversarial patches better adapt to different crowd scale perceptions, thereby enhancing the transferability between crowd counting models. Therefore, introducing a differentiable interpolation module allows adversarial patches to adapt to different model receptive fields. Through this specially designed module, the adversarial patch will randomly resize to execute attacks during optimization, forcing it to capture scale invariance between crowd counting models with different scale perceptions.

[0082] Specifically, given the original input image x and a randomly initialized adversarial patch δ, the adversarial patch is added to the image x according to formula (1) to form an adversarial sample x. adv Then, a difference module is proposed. x is randomly adjusted with probability p adv The magnitude of the data is then input into the crowd counting model to simulate changes in the receptive field. Therefore, adversarial examples can adapt to different scale perceptions and transfer better to different crowd counting models.

[0083] In step 102, the expression for the heatmap representing the attention of the corresponding model is:

[0084]

[0085] In formula (3), C is the number of people predicted by the crowd counting model, which is the summation of the output of the crowd counting model pixel by pixel. Let Z represent the pixel value at coordinates (i, j) of the k-th feature map of the model, Z is a hyperparameter used for global average pooling, and ReLU(·) represents the rectified linear function. This represents the density attention module.

[0086] It's important to note that various neural network models exhibit similar positional awareness when dealing with the same image. Different crowd counting models also display similar spatially distracted attention patterns at the same crowd locations. Therefore, by aggregating heatmaps, the shared attentional patterns of the models can be drawn to the adversarial patch region, thereby interfering with the target model's positional awareness. In this way, the generated adversarial patch can capture position-invariant features and execute more transferable attacks.

[0087] like Figure 3 As shown, in step 103, a loss function is introduced through a differentiable interpolation module and a heatmap representing the attention of the corresponding model, specifically including:

[0088] 1031. Scale-aware loss is obtained through a differentiable interpolation module;

[0089] 1032. The position-aware loss is obtained by using the heatmap corresponding to the attention of the model.

[0090] Since the objective of this invention is to attack the model and cause incorrect predictions, the crowd counting model should be forced to identify adversarial patches as crowds over a considerable length. Therefore, all values ​​output by the crowd counting model are aggregated, and a scale-aware loss without using any labels or real-world data is introduced.

[0091] The expression for the scale-aware loss is:

[0092]

[0093] In formula (4), f θi,j The pixel value at location coordinates (i, j) represents the predicted density map output by the crowd counting model. It is a differentiable interpolation module.

[0094] To draw the attention of the crowd counting model to the patch region, which could lead to incorrect estimations, we introduce a location-aware loss that directly increases the attention heat value on the patch region.

[0095] The expression for the location-aware loss is:

[0096]

[0097] In formula (5), S i,j The pixel value at position coordinates (i, j) of the heatmap representing the model's attention.

[0098] In step 103, the steps for optimizing adversarial examples specifically include:

[0099] 1033. The gradient-based iterative algorithm updates the adversarial examples until the loss function reaches the preset condition.

[0100] In embodiments of the present invention, such as Figure 3 As shown, by jointly optimizing the scale-aware loss L s and position perception loss L p Adversarial patches are generated. Specifically, a gradient-based iterative algorithm is used to optimize the adversarial patches. In each iteration, adversarial examples with initial adversarial patches are first generated at random locations; then, the crowd counting model's shared attention to the interpolated adversarial examples is enhanced, and the crowd counting model is largely misled into predicting them as human groups; finally, the adversarial patches are updated via backpropagation using the location matrix.

[0101] The loss function expression is as follows:

[0102]

[0103] In formula (6), For scale perception loss, This is due to the loss of location perception.

[0104] In this embodiment of the invention, the patch attributes are modified during the optimization process to maximize the loss function. The loss function is calculated using an iterative algorithm, and when the number of iterations reaches a predetermined threshold, the generation of adversarial examples needs to be stopped, and the final generated adversarial patch is stored.

[0105] In addition, such as Figure 1 As shown in (c), the adversarial examples generated using the adversarial attack method and apparatus provided in this embodiment of the invention can be used for adversarial training of crowd counting models, thereby improving the performance of existing crowd counting models. Furthermore, the adversarial training process using the adversarial examples generated in this embodiment of the invention does not iteratively solve a mini-maximum optimization problem; instead, all adversarial examples are generated at the beginning, which reduces the time required.

[0106] The adversarial patch provided in this invention can attack different crowd counting models, causing them to focus on perceptual regions at incorrect locations. Using the adversarial patch for adversarial training further enhances the crowd counting model's tolerance to scale and location perturbations. In other words, this adversarial patch strengthens the crowd counting model's generalization to multiple crowd scales and re-validates the model's perception by better focusing on the crowd itself under noise. In natural scenes, crowd size is more about the crowd region than the complex background; therefore, the adversarial patch provided in this invention can better generalize to unknown scenarios.

[0107] To implement the adversarial attack method provided by this invention, this invention also provides an adversarial attack device for crowd counting scenarios. For example... Figure 4As shown, the anti-attack device includes a processor 42 and a memory 41, and may further include communication components, sensor components, power supply components, multimedia components, and input / output interfaces as needed. The memory, communication components, sensor components, power supply components, multimedia components, and input / output interfaces are all connected to the processor 42. As previously mentioned, the memory 41 in the node device can be a static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, etc. The processor can be a central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), digital signal processing (DSP) chip, etc. Other communication components, sensor components, power supply components, multimedia components, etc., can all be implemented using common components in existing electronic devices, and will not be specifically described here.

[0108] On the other hand, in an adversarial attack device targeting crowd counting scenarios, the processor 42 reads the computer program in the memory 41 to perform the following operations:

[0109] By randomly scaling adversarial examples, a differentiable interpolation module is obtained;

[0110] Extract the attention representation of the crowd counting model for the differentiable interpolation module to obtain a heatmap of the corresponding attention representation model;

[0111] By introducing a loss function through a differentiable interpolation module and a heatmap representing the attention of the corresponding model, adversarial examples are optimized to attack the crowd counting model.

[0112] The effectiveness of the anti-attack method provided by this invention will be illustrated below through specific examples.

[0113] For attacks, six commonly used crowd counting models were selected: MCNN, CSRNet, CAN, BL, DM-Count, and SASNet. The mean absolute error (MAE) and mean squared error (MSE) were used to evaluate the crowd counting models. For attacks, higher MAE and MSE values ​​indicate stronger adversarial attacks; for model improvements, lower MAE and MSE values ​​indicate a better model.

[0114] For attacks, the adversarial attack method provided in this embodiment of the invention is compared with the only adversarial patch generation method (i.e., APAM) used for crowd counting. For model improvements, two adversarial training methods are compared: adversarial patches generated using APAM, PGD adversarial training, and three data augmentation methods: Cutout, Cutmix, and Augmix. Note that all methods use the same amount of additional data to train the crowd counting model.

[0115] For the attack, adversarial patches are initialized randomly with a fixed size and trained with a batch size of 1, iterating 100 times per iteration, with an attack step size of 0.01 and a maximum of 3 iterations. The position and orientation of the patches are randomly selected, which allows the adversarial patches provided by this invention to universally attack all images. We set the interpolation hyperparameters p, a, and b to 0.2, 0.9, and 1.1, respectively, and the position-aware loss weight λ to 10.

[0116] For model improvement, adversarial patches are first generated on each image in the original training set, and these patches are then mixed with the original training set to obtain a new training set (the ratio of adversarial samples to clean samples is 1:1). The crowd counting model is then trained using the new training set.

[0117] The performance of the adversarial attack method provided by this invention in the digital world was evaluated under both white-box and black-box settings. For white-box attacks, the adversary has complete knowledge of the target model and full access to it; for black-box attacks, the adversary has limited knowledge of the model and cannot directly access it. As for APAM, the published code was used, and the same settings were maintained for a fair comparison.

[0118] For white-box attacks, adversarial patches are generated using specific target models, and attacks are executed accordingly. As shown in Table 1 (diagonal), compared to APAM, the adversarial attack method provided by this invention achieves higher MAE and MSE in white-box settings of different models. Therefore, the adversarial attack method provided by this invention can generate adversarial patches with stronger white-box attack capabilities.

[0119] In the black-box setup, an adversarial patch based on a specific model is first generated, and then the attack is transferred to other models and its attack capability is tested. As shown in Table 1, we can draw the following observations:

[0120] 1. Compared with APAM, the adversarial attack method provided by this invention achieves stronger black-box attack capability by showing higher MAE and MSE values ​​in different models, and shows a greater advantage than APAMby (up to 1497.1% and 957.9% from DM count to BL).

[0121] 2. Adversarial attacks are difficult to transfer between multi-column (such as SASNet and MCNN) and single-column models. The likely reason is that multi-column models have a more complex architecture, with multiple branches and more information redundancy.

[0122] The attack results are shown in Table 1:

[0123] Table 1

[0124]

[0125] In Table 1, the first row shows the results for clean samples. Results on the diagonal are shown in white boxes, while other results are shown in black boxes. The adversarial attack method provided by this invention significantly outperforms APAM with higher MAE and MSE.

[0126] The performance of the adversarial patch generated by the adversarial attack method provided by this invention in the real physical world will be further evaluated below.

[0127] First, adversarial patches were generated using a CSRNet model, and then printed out. These patches were then affixed to flags or posters, and 96 photographs were taken. Different patch sizes (15cm, 15cm and 20cm, 20cm), distances (1.5m and 3m), and scenes (indoor and outdoor) were used in complex real-world scenarios. In each scene, six pairs of photographs were taken, with and without the adversarial patches. The generated adversarial patches were able to attack the crowd counting model.

[0128] like Figure 5 As shown, the generated adversarial patches appear very natural in the real world, but can introduce security issues during actual deployment. Figure 5 The counting model on the left, which did not engage in adversarial attacks, estimated the number of people to be 9, 9, and 7 respectively; the corresponding actual number of people was 7, 7, and 6; the prediction accuracy is high. Figure 5 The crowd counting model on the right after the adversarial attack estimated the number of people to be 109, 56, and 1332 respectively; the actual number of people was 6, 7, and 6 respectively; the difference between the estimated number of people is very large.

[0129] The following section will demonstrate the effectiveness of the adversarial attack method provided by this invention in improving the performance of crowd counting models. Specifically, it will evaluate the ability of the adversarial attack method provided by this invention to enhance the generalization ability of the crowd counting model on datasets and its robustness in complex background scenarios.

[0130] As shown in Table 2, due to its strong ability to identify crowds of different sizes, the crowd counting model trained using the adversarial attack method provided in this invention can significantly improve its generalization ability across datasets (MAE up to -26.3%, MSE up to -23.4%). The adversarial attack method provided in this invention also outperforms adversarial training baselines (such as APAM-AT and PAT). Furthermore, model generalization and data augmentation techniques (such as Cutout, Cutmix, and Augmix) degrade performance. The crowd counting model trained using the adversarial attack method provided in this invention exhibits better generalization ability (lower MAE and MSE) on the dataset.

[0131] Table 2

[0132]

[0133] To assess robustness in complex scenarios, the model performance was tested in scenarios with complex backgrounds using three test sets (including Distractors, Weather samples, and Negative samples).

[0134] like Figure 6 As shown, the crowd counting model trained using the adversarial attack method provided in this invention can improve the robustness of the three test sets (for interfering objects: -8.0% MAE and -4.2% MSE, for special weather samples: -2.1% MAE and -1.0% MSE, for negative samples: -28.5% MAE and -19.3% MSE; MAE is above, MSE is below).

[0135] Intuitively, using the adversarial patch provided in this invention for adversarial training can help the crowd counting model resist crowd noise and focus on real crowd patterns, thus making it more robust to negative samples. Figure 7 As shown, the attention of the crowd counting model trained using this invention can be more accurately focused on the human body region, while the density map of the original model (Vanilla Model) focuses on other interfering factors.

[0136] The following section will further analyze scale perception loss and position perception loss (i.e., L). s and L p The contribution of ) is discussed. Therefore, adversarial patches are generated from CSRNet with or without these two loss terms, and then transfer attacks are performed on other models on the dataset. As shown in Table 3, the perceptual loss L is added. s Afterwards, the MAE and MSE values ​​of all target models increased; simultaneously, a positional loss L was introduced. pSubsequently, the ability to transfer attacks also improved. The highest MAE and MSE values ​​were obtained when two losses were added. Therefore, the above experimental results demonstrate the effectiveness of scale-aware loss and position-aware loss in improving the transferability of attacks.

[0137] Table 3

[0138]

[0139] The adversarial attack method provided in this invention learns model-invariant features by utilizing model scale-aware loss and position-aware loss. Furthermore, adversarial patches can be used to improve the performance of crowd counting models through adversarial training. The effectiveness of this adversarial attack method is further verified through experiments. Experiments in the digital and physical worlds demonstrate that the adversarial attack method provided in this invention achieves state-of-the-art performance. Using the adversarial patches provided in this invention for adversarial training can significantly improve the performance of crowd counting models.

[0140] The foregoing detailed description of the adversarial attack method and apparatus for crowd counting scenarios provided by this invention is intended to clarify that any obvious modifications made by those skilled in the art without departing from the essential content of this invention will constitute an infringement of the patent rights of this invention and will incur corresponding legal liability.

Claims

1. An adversarial attack method for crowd counting scenarios, characterized in that... Includes the following steps: The adversarial patch is randomly initialized, and the position matrix is ​​randomly generated to obtain the adversarial sample; wherein, the adversarial sample consists of a normal sample, the adversarial patch, and the position matrix; The size of the adversarial sample is randomly adjusted with the probability of the interpolation operation to obtain a differentiable interpolation module; the differentiable interpolation module enables the adversarial patch to adapt to the receptive field of different crowd counting models; wherein, the size of the adversarial patch is randomly adjusted so as to perform an attack during the optimization process and capture the scale invariance between crowd counting models with different scale perceptions; Extract the attention representation of the crowd counting model for the differentiable interpolation module to obtain a heatmap of the corresponding attention representation model; The scale-aware loss is obtained through a differentiable interpolation module; the position-aware loss is obtained through a heatmap representing the model's attention; wherein, the expression for the scale-aware loss is: In the above formula, fθi,j The predicted density map output by the population counting model is located at coordinates ( i , j Pixel value at ) It is a differentiable interpolation module; The expression for the location-aware loss is: In the above formula, Si,j A heatmap representing the model's attention at location coordinates ( i , j The pixel value at (). A loss function is introduced to optimize adversarial examples, thereby attacking the crowd counting model; wherein, the expression of the loss function is: = max + λ In the above formula, For scale perception loss, For scale perception loss, This is due to the loss of location perception.

2. The counter-attack method as described in claim 1, characterized in that, The optimized adversarial examples specifically include: Gradient-based iterative algorithms update adversarial examples until the loss function reaches a preset condition.

3. The counter-attack method as described in claim 1, characterized in that, The expression for the differentiable interpolation module is: In the above formula, p It represents the probability of performing an interpolation operation. a , b These are the lower and upper limits of the image scaling ratio, respectively. This indicates upsampling or downsampling operations.

4. The counter-attack method as described in claim 1, characterized in that, The expression for the heatmap representing the attention of the corresponding model is: In the above formula, C is the population size predicted by the population counting model. The model number is represented by the first... k Zhang's feature map is located at position coordinates ( i , j The pixel value at ) Z These are hyperparameters used for global average pooling. Represents the linear rectified function. This represents the density attention module.

5. An adversarial attack device for crowd counting scenarios, characterized in that... It includes a processor and a memory, wherein the processor reads a computer program from the memory for executing the anti-attack method according to any one of claims 1 to 4.

Images