A general person re-identification attack method based on reduced style features

Abstract

The present invention discloses a universal person re-identification attack method based on reducing style features. First, an intermediate layer module is constructed based on random sample images. Then, the gradient of the attack result graph on the proxy model is calculated to obtain the image content feature information on the proxy model. The gradient of the attack result graph on the style feature model is calculated to obtain the image style feature information on different style feature models. Finally, the loss function is calculated, and the above operations are repeated for training to obtain the final attack result graph. The present invention extracts the changes of the attack result graph in different style feature models by randomly generating different style feature models and superimposes them, thereby eliminating the style dependence on the proxy model. Compared with existing pedestrian re-identification attack methods, the universality is improved, and it is less affected by style features. The attack result graph contains much more feature information of the image itself than the style feature information.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and is mainly applied to pedestrian re-identification attack methods based on deep learning. Specifically, it relates to a universal pedestrian re-identification attack method based on reducing style features. Background Art

[0002] In the past five years, the rise of deep learning technology has driven rapid development in the field of computer vision. Significant progress has been made in tasks such as license plate recognition, object detection, face recognition, image style transfer, and image classification.

[0003] Person re-identification (Re-ID) is a key task in computer vision. Its goal is to extract pedestrian features from images acquired by multiple cameras, enabling cross-camera pedestrian tracking and recognition. This technology plays a key role in applications such as criminal suspect tracking and identifying people entering and leaving the scene.

[0004] In recent years, with the continuous development of deep learning technology, pedestrian re-identification has made significant progress. The application of deep neural networks has made it possible to accurately identify pedestrians under different viewing angles and lighting conditions. This provides a powerful tool for the security field, helping to improve the effectiveness of surveillance systems.

[0005] While deep learning has made significant progress, it also faces challenges. Its robustness and security have become a focus of academic attention. Some researchers have shown that adding tiny, invisible pixel perturbations to images of giant pandas is sufficient to cause deep learning-based classifiers to misidentify them as gibbons. This adversarial attack reveals the sensitivity of deep learning models to subtle changes, raising concerns about their robustness.

[0006] Research on attack algorithms is crucial for improving the robustness of person re-identification algorithms. By gaining a deeper understanding of and addressing adversarial attacks, we can provide more reliable defense mechanisms for person re-identification systems. This research not only helps improve the robustness of deep learning models but also provides important guidance for addressing potential security threats.

[0007] Most existing attack techniques are performed on a single target model. However, when using trained attack samples to attack another target model, they often do not have good attack performance. This also shows that the attack technique is only applicable to the corresponding target model and is not universal. One reason for this low universality is that the style features of different models vary, resulting in some style feature information being carried during the attack, while other models have different style features. Therefore, this patent improves the universality of attack samples by reducing the style feature information of the attack samples. Summary of the Invention

[0008] Aiming at the task of person re-identification, this paper proposes a universal person re-identification attack method based on reducing style features.

[0009] A general person re-identification attack method based on reduced style features includes the following steps:

[0010] Step 1: Construct an intermediate layer module based on random sample images;

[0011] Step 2: Calculate the gradient of the attack result graph on the proxy model to obtain the image content feature information on the proxy model;

[0012] Step 3: Calculate the gradient of the attack result image on the style feature model to obtain the image style feature information on different style feature models;

[0013] Step 4: Calculate the loss function and repeat steps 2 and 3 for training to obtain the final attack result graph;

[0014] Step 5: Implement person re-identification attack through attack result graph;

[0015] The final attack result image is used as the input of the target pedestrian re-identification model to reduce the recognition accuracy of the target pedestrian re-identification model and complete the attack.

[0016] Furthermore, the specific method of step 1 is as follows;

[0017] Randomly select a pedestrian image from the pedestrian re-identification dataset as the original image x, and randomly select a sample image s, and calculate the parameters of the middle layer:

[0018] μ=β(λμ x +(1-λ)μ s ),

[0019] σ=γ(λσ x +(1-λ)σ s ),

[0020] where μ x , σ x are the mean and variance of the original image x, μ s , σ s are the mean and variance of the random sample image s, β is the mean coefficient, γ is the variance coefficient, λ is the linear weight between the original image and the random sample image, β and γ are random values ​​in [0,2], and λ is a random value in [0,0.02].

[0021] The corresponding middle layer is generated by the following formula:

[0022]

[0023] Furthermore, the specific method of step 2 is as follows;

[0024] The proxy model is used to retain the content feature information of the image itself. Since the way to improve versatility is to reduce the influence of style features, in other words, it is necessary to retain enough image content feature information. The more it retains, the more universal the attack result graph will be.

[0025] The proxy model uses ResNet-50 and can be expressed as:

[0026]

[0027] F2 is the encoder, which is the first bottleneck block of the ResNet-50 model. F1 is the decoder, which is the part after the first bottleneck block.

[0028] The attack result graph x adv Initialize to the original image x, and convert x adv As the proxy model input, calculate F(x adv ) for x adv Gradient t represents the number of training rounds, t=0, 1, 2, ...T-1, and T represents the total number of training rounds.

[0029] Furthermore, the specific method of step 3 is as follows;

[0030] Different style feature models are constructed based on pictures with different style features. The style feature information of the image is extracted and superimposed through different style feature models to eliminate the dependency of the feature result graph on the parameters of a single specific style feature model.

[0031] By inserting an intermediate layer module in the middle of the pedestrian re-identification proxy model to represent the style feature model trained with different style feature datasets, the influence of different style feature models on the attack result graph can be obtained and constrained.

[0032] The style feature model consists of an encoder part, an intermediate layer module, and a decoder part; the encoder and decoder parts use the ResNet-50 model as the basis, and the intermediate layer is generated based on random sample images.

[0033] Specifically, the intermediate layer is inserted after the first bottleneck block (bottelneck module) of the ResNet-50 model.

[0034] The style feature model can be expressed as:

[0035]

[0036] The attack result graph x adv As input to the style feature model, calculate x adv The gradient g i , i=0,1,2,...N-1, N represents the total number of style feature models.

[0037] Furthermore, the specific method of step 4 is as follows;

[0038] The loss function includes the loss of the proxy model and the loss of the style feature model. The style feature model includes the losses of multiple style feature models of different styles. The losses of different style feature models are used to constrain the style loss in the proxy model.

[0039] The loss function can be expressed as:

[0040]

[0041] Where L(·,·) represents the mean squared error function, δ represents the perturbation superimposed on the original image during training, and y represents the pedestrian number in the original image. By maximizing the loss function, both the style feature model loss and the proxy model loss are increased. During training, this method makes the latter significantly greater than the former, thereby improving the versatility of the attack result images.

[0042] The calculation method is as follows:

[0043] In each round of training, N different style feature models are calculated separately to obtain the gradient of the attack result graph on the different style feature models, and then superimposed on the gradient of the attack result graph on the proxy model obtained in step 2, which can be expressed as follows:

[0044]

[0045] in Represents the superimposed gradient, which represents the gradient of the style feature model to the attack result graph.

[0046] Since it is obtained by superposition of multiple style feature models, Normalization.

[0047]

[0048] in represents the normalized gradient.

[0049] By The gradient g obtained in the previous round t Weighted superposition is performed, and the gradient g0 is initialized to 0. This is to retain the influence of the results of the previous rounds, rather than only deriving results based on the current round. This approach can make the training process smoother.

[0050] The formula is as follows:

[0051]

[0052] η represents the weight of the previous round of gradient.

[0053] Pass the gradient calculation results to the attack result graph.

[0054] x adv '=x adv +α·sign(g t+1 )

[0055] Where α is the superposition coefficient, sign(·) represents the sign function, and is expressed as follows:

[0056]

[0057] Iterate T times to get the final attack result graph.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] We analyze the mechanism of attack transferability from the perspective of style features and improve versatility by constructing different style feature models through intermediate layers. By randomly generating different style feature models, we extract the variations of the attack result graph across these different style feature models and superimpose them, eliminating the style dependency of the proxy model. Compared to existing person re-identification attack methods, this approach improves versatility and is less affected by style features. The attack result graph contains much more information about the image itself than about the style features. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a structural diagram of a style feature model according to an embodiment of the present invention;

[0061] Figure 2 This figure shows the experimental results of the embodiment of the present invention. DETAILED DESCRIPTION

[0062] The technical solutions of the present invention are further described below in conjunction with the accompanying drawings and embodiments.

[0063] The image size used in this embodiment is 128x256. A general pedestrian re-identification attack method based on reducing style features includes the following steps:

[0064] Step 1: Construct an intermediate layer module based on random sample images;

[0065] Randomly select a pedestrian image from the dataset as the original image x, and randomly select a sample image s, and calculate the parameters of the middle layer:

[0066] μ=β(λμ x +(1-λ)μ s),

[0067] σ=γ(λσ x +(1-λ)σ s ),

[0068] where μ x , σ x are the mean and variance of the original image x, μ s , σ s are the mean and variance of the random sample image s, β is the mean coefficient, γ is the variance coefficient, λ is the linear weight between the original image and the random sample image, β and γ are random values ​​in [0,2], and λ is a random value in [0,0.02].

[0069] The corresponding middle layer is generated by the following formula:

[0070]

[0071] Step 2: Calculate the gradient of the attack result graph on the proxy model to obtain the image content feature information on the proxy model;

[0072] The proxy model uses ResNet-50 and can be expressed as:

[0073]

[0074] F2 is the encoder, which is the first bottleneck block of the ResNet-50 model. F1 is the decoder, which is the part after the first bottleneck block.

[0075] The attack result graph x adv Initialize to the original image x, and convert x adv As the proxy model input, calculate F(x adv ) for x adv Gradient t represents the number of training rounds, t=0, 1, 2, ...T-1, T represents the total number of training rounds, which is set to 50 in this embodiment.

[0076] Step 3: Calculate the gradient of the attack result graph on multiple different style feature models to obtain image style feature information on different style feature models;

[0077] Different style feature models are constructed based on images with different style features. Since the model will superimpose the gradients of image content and image style on the attack result image during training, the starting point is to achieve a universal pedestrian re-identification attack method by reducing the influence of image style.

[0078] By extracting and superimposing the style feature information of the image through different style feature models, the dependence of the feature result map on the parameters of a single specific style feature model is eliminated, thereby achieving the purpose of improving the versatility of pedestrian re-identification.

[0079] like Figure 1 As shown in the figure, by inserting an intermediate layer module in the middle of the pedestrian re-identification proxy model to represent the style feature model trained with different style feature datasets, the influence of different style feature models on the attack result graph is obtained, and thus it can be constrained.

[0080] The style feature model consists of an encoder part, an intermediate layer module, and a decoder part; the encoder and decoder parts use the ResNet-50 model as the basis, and the intermediate layer is generated based on random sample images.

[0081] Specifically, the intermediate layer is inserted after the first bottleneck block (bottelneck module) of the ResNet-50 model.

[0082] The style feature model can be expressed as:

[0083]

[0084] The attack result graph x adv As input to the style feature model, calculate x adv Gradient i=0, 1, 2, ... N-1, where N represents the total number of style feature models, which is set to 10 in this embodiment.

[0085] Step 4: Calculate the loss function and repeat steps 2 and 3 for training to obtain the final result graph;

[0086] The loss function can be expressed as:

[0087]

[0088] Where L(·,·) represents the mean squared error function, δ represents the perturbation superimposed on the original image during training, and y represents the pedestrian number in the original image. By maximizing the loss function, both the style feature model loss and the proxy model loss are increased. During training, this method makes the latter significantly greater than the former, thereby improving the versatility of the attack result images.

[0089] The calculation method is as follows:

[0090] In each round of training, N different style feature models are calculated separately to obtain the gradient of the attack result graph on the different style feature models, and then superimposed on the gradient of the attack result graph on the proxy model obtained in step 2, which can be expressed as follows:

[0091]

[0092] in Represents the superimposed gradient, which represents the gradient of the style feature model to the attack result graph.

[0093] Since it is obtained by superposition of multiple style feature models, Normalization.

[0094]

[0095] in represents the normalized gradient.

[0096] By The gradient g obtained in the previous round t Weighted superposition is performed, and the gradient g0 is initialized to 0. This is to retain the influence of the results of the previous rounds, rather than only deriving results based on the current round. This approach can make the training process smoother.

[0097] The formula is as follows:

[0098]

[0099] η represents the weight of the previous round of gradients, and in this embodiment it is set to 0.1.

[0100] Pass the gradient calculation results to the attack result graph.

[0101] x adv '=x adv +α·sign(g t+1 )

[0102] Wherein, α is the superposition coefficient, which is 8 / 255 in this embodiment, and sign(·) represents the sign function, which is expressed as follows:

[0103]

[0104] Iterate T times to get the final attack result graph.

[0105] Step 5: Implement person re-identification attack through attack result graph

[0106] The final attack result image is used as the input of the target pedestrian re-identification model to reduce the recognition accuracy of the target pedestrian re-identification model and complete the attack.

[0107] The experimental data are shown in the following table:

[0108] Proxy Model Attack Methods ResNet-50 WRN101 DN121 ResNet-50 I / +Ours 100 / 100 80.8 / 97.1 83.0 / 97.8 WRN101 I / +Ours 88.6 / 98.7 100 / 100 77.7 / 94.2 DN121 I / +Ours 87.0 / 99.4 74.7 / 98.0 100 / 100

[0109] Figure 2The figure shows the experimental results of the embodiment. It can be seen that after using this method on the original method, the attack performance has been significantly improved when attacking other pedestrian re-identification models.

[0110] The above description is a further detailed description of the present invention in conjunction with specific / preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art of the present invention may make various substitutions or modifications to the described embodiments without departing from the scope of the present invention, and such substitutions or modifications should be considered to fall within the scope of protection of the present invention.

[0111] Parts of the present invention that are not described in detail belong to the common knowledge of those skilled in the art.

Claims

1. A general person re-identification attack method based on reduced style features, characterized by: The following steps are involved: Step 1: Construct an intermediate layer module based on random sample images; Step 2: Calculate the gradient of the attack result graph on the proxy model to obtain the image content feature information on the proxy model; Step 3: Calculate the gradient of the attack result image on the style feature model to obtain the image style feature information on different style feature models; Step 4: Calculate the loss function and repeat steps 2 and 3 for training to obtain the final attack result graph; The loss function includes the loss of the proxy model and the loss of the style feature model. The style feature model includes the losses of multiple style feature models of different styles. The losses of different style feature models are used to constrain the style loss in the proxy model. The loss function can be expressed as: where L(·,·) represents the mean square error function, represents the perturbation superimposed on the original image during training, and y represents the pedestrian number in the original image. By maximizing the loss function, the style feature model loss and the proxy model loss are increased. During training, this method will make the latter much larger than the former, thereby improving the versatility of the attack result image. The calculation method is as follows: In each round of training, N different style feature models are calculated separately to obtain the gradient of the attack result graph on the different style feature models, and then superimposed on the gradient of the attack result graph on the proxy model obtained in step 2, which can be expressed as follows: in represents the superimposed gradient, which represents the gradient of the style feature model to the attack result graph; Since it is obtained by superposition of multiple style feature models, Normalization; in represents the normalized gradient; By Its gradient obtained in the previous round Perform weighted superposition, gradient Initialized to 0, this is to retain the influence of the results of the previous rounds, rather than only drawing results based on the results of the current round, which can make the training process smoother; The formula is as follows: Represents the weight of the previous round of gradient; Pass the gradient calculation results to the attack result graph; in, is the superposition coefficient, It represents the symbolic function and is expressed as: Iterate T times to get the final attack result graph; Step 5: Implement person re-identification attack through attack result graph; The final attack result image is used as the input of the target pedestrian re-identification model to reduce the recognition accuracy of the target pedestrian re-identification model and complete the attack.

2. The general person re-identification attack method based on reduced style features according to claim 1, characterized in that: Step 1: The specific method is as follows; Randomly select a pedestrian image from the pedestrian re-identification dataset as the original image x, and randomly select a sample image s, and calculate the parameters of the middle layer: in are the mean and variance of the original image x, are the mean and variance of random sample images s, is the mean coefficient, is the coefficient of variance, is the linear weight between the original image and the random sample image, A random value of Random value of The corresponding middle layer is generated by the following formula: 。 3. The general person re-identification attack method based on reduced style features according to claim 1, characterized in that: Step 2: The specific method is as follows; Using proxy models to retain the content feature information of the image itself. Since the way to improve versatility is to reduce the influence of style features, in other words, it is necessary to retain enough image content feature information. The more content feature information is retained, the more universal the attack result graph will be. The proxy model uses ResNet-50 and can be expressed as: F2 is the encoder, which is the first bottleneck block of the ResNet-50 model; F1 is the decoder, which is the part after the first bottleneck block; The attack result graph Initialize to the original image x, As the proxy model input, calculate right Gradient ; t represents the number of training rounds, t=0, 1, 2, ...T-1, T represents the total number of training rounds.

4. A general person re-identification attack method based on reduced style features according to claim 2 or 3, characterized in that: Step 3: The specific method is as follows; Different style feature models are constructed based on images with different style features. The style feature information of the image is extracted and superimposed through different style feature models to eliminate the dependency of the feature result graph on the parameters of a single specific style feature model. By inserting an intermediate layer module into the pedestrian re-identification proxy model to represent the style feature models trained with different style feature datasets, the influence of different style feature models on the attack result graph can be obtained and constrained. The style feature model consists of an encoder, an intermediate layer module, and a decoder. The encoder and decoder use the ResNet-50 model as the basis, and the intermediate layer is generated based on random sample images. Specifically, the intermediate layer is inserted after the first bottleneck block of the ResNet-50 model; The style feature model can be expressed as: The attack result graph As input to the style feature model, calculate right Gradient , i=0,1,2,...N-1, N represents the total number of style feature models.

Images