A black-box adversarial attack method for evaluating robustness of intelligent security system

Abstract

The application belongs to the field of confrontation attack, and relates to a black-box confrontation attack method for evaluating the robustness of an intelligent security system, comprising the following steps: obtaining a data set, and training a substitute model by using the data set; constructing a neighborhood image of an input image by using a neighborhood data point sampling algorithm; inputting the neighborhood image after preprocessing into the substitute model to obtain a prediction result and an intermediate layer feature; calculating an average gradient according to the prediction result and the intermediate layer feature by using a spatial momentum algorithm; calculating spatial momentum according to the average gradient, and calculating an adversarial perturbation according to the spatial momentum; combining the input image and the adversarial perturbation to generate an adversarial sample; repeating the above steps until a preset iteration number is reached; attacking the intelligent security system by using the adversarial sample to verify the robustness of the system; the application combines the neighborhood sampling algorithm and the spatial momentum algorithm to correct and stabilize the gradient update direction; and the intermediate layer feature is combined in a loss function, so that the attack of the model is more targeted, and the attack success rate is improved.

Description

Technical Field

[0001] This invention belongs to the field of adversarial attacks, specifically relating to a black-box adversarial attack method for evaluating the robustness of intelligent security systems. Background Technology

[0002] In recent years, the development of intelligent security technology has become an important direction in the field of social security. These systems utilize advanced computer vision and deep learning technologies to perform real-time monitoring, anomaly detection, and threat identification in various environments, thereby improving security and response capabilities. Intelligent security systems play an irreplaceable role in protecting public places, commercial spaces, and private residences. With the widespread application of intelligent security systems, the demand for their security and robustness is becoming increasingly prominent. Developing effective black-box countermeasures against attackers to evaluate the robustness of intelligent security systems against attacks is of significant theoretical and practical importance.

[0003] Deep learning-based adversarial attack techniques have demonstrated excellent attack performance in black-box models, and can be divided into methods based on improving gradient calculation by incorporating momentum terms and methods based on diversified input samples. Methods based on diversified input samples mainly expand the dataset quickly and effectively by applying certain transformation operations to the original input samples, thereby improving the success rate of adversarial black-box attacks. However, these methods only consider solving the overfitting problem and do not consider how to effectively overcome local extrema during gradient updates. To address this issue, methods based on improving gradient calculation by incorporating momentum terms have been proposed. These methods improve the momentum accumulation mechanism based on the momentum method. Compared with traditional gradient descent algorithms, these methods typically converge to the optimal solution of the loss function faster during training and reduce local oscillations, effectively improving the training speed and performance of the model, thereby increasing the success rate of adversarial attacks.

[0004] However, the above methods still have the following problems: 1) They do not take into account the simultaneous correction and stabilization of gradient update direction, which makes the gradient update not proceed in an effective direction and easily get stuck in local extrema, thus making the attack success rate low; 2) They do not take into account the impact of intermediate layer features on the overall decision of the model, which makes the attack less targeted. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, this invention proposes a black-box adversarial attack method for evaluating the robustness of intelligent security systems, comprising:

[0006] S1. Obtain a dataset D similar to the training intelligent security system, use dataset D to train an alternative model, and obtain the trained alternative model.

[0007] S2. Obtain the input image from dataset D, and construct the neighborhood image of the input image using the neighborhood data point sampling algorithm;

[0008] S3. Preprocess the neighborhood image, input the preprocessed image into the trained substitution model, and obtain the prediction result and intermediate layer output features.

[0009] S4. The fusion spatial momentum algorithm calculates the average gradient based on the prediction results and the output characteristics of the intermediate layer.

[0010] S5. Calculate spatial momentum based on the average gradient, and calculate the resistance perturbation based on the spatial momentum;

[0011] S6. Combine the input image with adversarial perturbations to generate adversarial examples; if the preset maximum number of iterations T is reached, the final adversarial example is obtained; otherwise, return to step S2 and construct the neighborhood image of the current adversarial example.

[0012] S7. Use the final adversarial sample to attack the intelligent security system and verify the system's robustness.

[0013] Obtaining a dataset similar to that used for training the intelligent security system involves interacting with the input and output interfaces of the intelligent security system and repeatedly querying and observing the output of the intelligent security system to obtain a dataset similar to that used for training the intelligent security system.

[0014] The neighborhood image for constructing adversarial examples includes:

[0015]

[0016] in, Let U(t) represent the adversarial example generated in the t-th iteration, β represent the boundary radius of the neighborhood sampling of the adversarial example, and U(·) represent the adversarial example generated in the t-th iteration. d ,(·) d ) represents a uniform distribution in d-dimensional space, and ε represents the perturbation threshold. Let represent the i-th neighborhood image of the adversarial sample generated in the t-th iteration, where t is the iteration number.

[0017] Preprocessing the neighborhood image includes: dividing the neighborhood image into multiple sub-images at different locations; and applying a random left-right flip transformation to all sub-images with probability q.

[0018] The fusion of spatial momentum algorithms and the calculation of the average gradient based on prediction results and intermediate layer output features include: obtaining the feature momentum γ of the t-th iteration. t According to the characteristic momentum γ t Calculate the characteristic momentum γ of the (t+1)th iteration using the prediction results of the (t+1)th iteration. t+1 According to the characteristic momentum γ t+1The loss function value is calculated using the output features of the intermediate layer in the (t+1)th iteration, and the average gradient is calculated based on the loss function value.

[0019]

[0020] Where, λ i λ represents the weight values ​​for gradient updates. i = 1 / N, where N is the number of neighboring images, and H(·) represents preprocessing. Let y represent the i-th neighborhood image of the adversarial example generated in the t-th iteration. true Let represent the true label corresponding to the input image, θ represent the hyperparameters of the substitution model, and J(·) represent the loss function. Let represent the average gradient generated in the (t+1)th iteration, where t is the iteration number and i is the index of the neighborhood image.

[0021] Calculate the characteristic momentum γ of the (t+1)th iteration. t+1 This includes: calculating the gradient of the k-th layer of the surrogate model in the (t+1)-th iteration based on the prediction results of the surrogate model. according to Calculate the characteristic momentum γ of the (t+1)th iteration. t+1 :

[0022]

[0023] Where λ is the weight, ||·||1 is the L1 norm, and γ t Let be the characteristic momentum of the t-th iteration.

[0024] The calculation of the loss function value includes:

[0025]

[0026] Where ⊙ denotes matrix multiplication, and M is the number of features output by the k-th layer of the substitution model. This is used to replace the j-th feature output by the k-th layer in the (t+1)-th iteration of the model.

[0027] Calculating spatial momentum based on the average gradient includes:

[0028]

[0029] Among them, g t Let g be the spatial momentum of the t-th iteration. t+1 Let μ be the space momentum in the (t+1)th iteration, and μ be the momentum decay factor. Let be the average gradient generated in the (t+1)th iteration, where t is the iteration number and ||·||1 is the L1 norm.

[0030] Generate adversarial examples by combining the input image with adversarial perturbations. include:

[0031]

[0032] in, For the adversarial example generated in the (t+1)th iteration, sign(g) t+1 ) represents the adversarial perturbation, α is the update step size, sign(·) is the sign function, x is the input image, and g t+1 Let be the spatial momentum of the (t+1)th iteration, where t is the iteration number.

[0033] The beneficial effects of this invention are as follows:

[0034] 1. This invention combines a neighborhood data point sampling algorithm and a spatial momentum algorithm, which not only corrects the gradient update direction in each iteration, ensuring the gradient update moves in an effective direction, but also effectively stabilizes the gradient update direction, making it less prone to getting trapped in local extrema, thereby improving the attack success rate. 2. This invention combines the model's final prediction results and intermediate layer features in the loss function, further considering the impact of intermediate layer features on the model's overall decision-making, making the model's attacks more targeted and improving the attack success rate. 3. This invention can generate high-quality, highly realistic adversarial examples that simulate the challenges in real-world attack scenarios. This realism makes the evaluation more accurate and can more realistically reflect the performance of intelligent security systems when facing black-box attacks, providing valuable guidance for system design and improvement. Attached Figure Description

[0035] Figure 1 A flowchart of a black-box adversarial attack method for evaluating the robustness of an intelligent security system, provided in an embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram of space momentum update provided for an embodiment of the present invention;

[0037] Figure 3 This is a comparative diagram of different attack methods provided in the embodiments of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] This invention proposes a black-box adversarial attack method (ESMI-FGSM) for evaluating the robustness of intelligent security systems, such as... Figure 1 As shown, the method includes the following:

[0040] S1: Obtain a dataset similar to the one used to train the intelligent security system, and use the dataset to train an alternative model to obtain the trained alternative model, which is usually a white-box model.

[0041] S11: Interact with the input / output interfaces of the intelligent security system to obtain a dataset similar to the one used to train the intelligent security system by repeatedly querying or observing the output of the intelligent security system.

[0042] S12: Use the dataset to train an alternative model of an intelligent security system, and generate adversarial samples needed for attacks on this model.

[0043] The purpose of the attack is to find an adversarial sample. satisfy And it can successfully mislead the model into making an incorrect prediction for this sample, where ‖·‖ p L represents p The norm will limit adversarial perturbations to a certain threshold ε.

[0044] S2: Obtain the input image and construct its neighborhood image using a neighborhood data point sampling algorithm. If adversarial examples are obtained in subsequent iterations, replace the input image in this step with the adversarial examples and construct their neighborhood images. Subsequent steps follow the same principle. The specific formula is as follows:

[0045]

[0046] in, Let U represent the input image or adversarial example, R represent the boundary radius of the neighborhood sampling of the input image or adversarial example, and U(·) represent the boundary radius of the neighborhood sampling of the input image or adversarial example. d ,(·) d ) represents a uniform distribution in d-dimensional space. ε represents the i-th neighboring image of the input image or adversarial sample, and ε represents the perturbation threshold. If it is too small, the generated adversarial sample will not mislead the classifier to produce incorrect results. If it is too large, it will destroy the semantic information of the image, distort the image, and lose the meaning of the adversarial attack.

[0047] S3: Preprocess the neighborhood image, input the preprocessed image into the trained substitution model, and obtain the prediction results and intermediate layer output features;

[0048] Preprocessing the neighborhood image involves segmenting the input image into multiple sub-images at different locations. For example, if the target model requires the input image to have a pixel value of 299×299×3, a cropping range needs to be selected first. This range should be chosen based on the model's characteristics and the image's content to ensure that the cropped sub-images still contain enough information for the model to recognize. For example, a range like [270, 299] can be selected, and multiple random cropping operations can be performed within this range. The advantage of this is that multiple different sub-images can be obtained from the original image, each representing a part of the original image. After cropping, the sub-images also need to be padded to ensure that their pixel values ​​reach 299×299×3. It is important to note that the cropping range should not be too large. If the cropping range is too large, the resulting adversarial examples may excessively distort the image.

[0049] To avoid overfitting during training, all subgraphs are subjected to a random left-right flip transformation with a certain probability (usually set to 0.5).

[0050] S4. The fusion spatial momentum algorithm calculates the average gradient based on the prediction results and the output characteristics of the intermediate layer.

[0051] Obtain the characteristic momentum γ of the t-th iteration t According to the characteristic momentum γ t Calculate the characteristic momentum γ of the (t+1)th iteration using the prediction results of the (t+1)th iteration. t+1 According to the characteristic momentum γ t+1 The loss function value is calculated using the output features of the intermediate layer in the (t+1)th iteration, and the average gradient is calculated based on the loss function value.

[0052]

[0053]

[0054] Where, λ i , λ p λ represents the weight values ​​for gradient updates. i =1 / N, λ p = 1 / P, where N is the number of neighborhood images, P is the number of sub-images in the preprocessed neighborhood image, and H p (·) indicates preprocessing of the neighborhood image. Let y represent the i-th neighborhood image of the adversarial example generated in the t-th iteration. true Let represent the true label corresponding to the input image, θ represent the hyperparameters of the model, and J(·) represent the loss function. γ represents the average gradient of the (t+1)th iteration, where t is the iteration number. t+1 Let i be the feature momentum of the (t+1)th iteration, and let i be the index of the neighborhood image.

[0055] The formula for calculating characteristic momentum is:

[0056]

[0057] in, Let λ be the gradient of the k-th layer in the (t+1)-th iteration, and let λ be the weight, which is usually set to 1 to 1.5.

[0058] The formula for calculating the gradient of the k-th layer in the (t+1)-th iteration is:

[0059]

[0060] Where L is the original loss function between the prediction result and the true result of the alternative model, and F k Let P represent the output feature of the k-th layer in the (t+1)-th iteration, where P is the number of subgraphs segmented.

[0061] The formula for calculating the loss function is:

[0062]

[0063] Where ⊙ denotes matrix multiplication, and M is the number of features output by the k-th layer of the substitution model. This is used to replace the j-th feature output by the k-th layer in the (t+1)-th iteration of the model.

[0064] S5. Calculate spatial momentum based on the average gradient, and calculate the resistance to perturbations based on the spatial momentum:

[0065] Calculating spatial momentum based on the average gradient includes:

[0066]

[0067] Among them, g t Let g be the spatial momentum of the t-th iteration. t+1 Let μ be the space momentum in the (t+1)th iteration, and μ be the momentum decay factor, which is usually set to 1. Let be the average gradient of the (t+1)th iteration.

[0068] S6. Combine the input image with adversarial perturbations to generate adversarial examples; if the preset maximum number of iterations T is reached, the final adversarial example is obtained; otherwise, return to step S2 and construct the neighborhood image of the current adversarial example.

[0069] A schematic diagram of spatial momentum update is shown below. Figure 2As shown, after each iteration, the spatial momentum is generated based on the average gradient information to update the adversarial perturbation. After T attacks on the model, the final adversarial perturbation is obtained. The image of the original input model is combined with the final adversarial perturbation to generate an adversarial example. This adversarial example can successfully deceive the target model, while keeping the value of the adversarial perturbation below a certain threshold. These perturbations are negligible to human vision. The formula for combining them is:

[0070]

[0071] in, For the adversarial example generated in the (t+1)th iteration, sign(g) t+1 ) represents the adversarial perturbation, α is the update step size, sign(·) is the sign function, x is the input image, and g t+1 Let be the spatial momentum generated in the (t+1)th iteration, where t is the iteration number.

[0072] S7. Use adversarial sample attacks on the intelligent security system to verify the system's robustness.

[0073] Attacking intelligent security systems involves using adversarial examples generated on alternative models to attack the actual intelligent security system. The robustness of the system is verified by its prediction and recognition performance. If the prediction and recognition are correct, the system is robust; otherwise, the system is not robust.

[0074] To verify that the proposed method ESMI-FGSM can stabilize the gradient update direction and improve the success rate of adversarial attacks, thereby better verifying the robustness of intelligent security systems, the enhanced spatial momentum fast gradient sign method (ESMI-FGSM), momentum iterative fast gradient sign method (MI-FGSM), Nesterov iterative fast gradient sign method (NI-FGSM), variance-adjusted momentum iterative fast gradient sign method (VMI-FGSM), and variance-adjusted Nesterov iterative fast gradient sign method (VNI-FGSM) were tested on the ILSVRC2012 validation set of the ImageNet dataset, using different models. The attack success rate was used to evaluate the algorithm's performance in the experiments. The models included were Inception-v3 (Inc-v3), Inception... -v4 (Inc-v4), Inception-Resnet-v2 (IncRes-v2), Inception-v3_ens3 (Inc-v3_ens3), Inception-v3_ens4 (Inc-v3_ens4), InceptionResnet-v2_ens (IncRes-v2_ens); The ILSVRC2012 validation set is a widely used image recognition dataset containing 1.2 million images from 1000 different categories. These images are precisely labeled, with one image selected from each category, for a total of 1000 images as the dataset. Experimental results are as follows... Figure 3 As shown in the figure, * represents white-box attacks, and bold data represents the data with the best attack performance. It can be observed that ESMI-FGSM consistently outperforms other comparative methods, achieving the highest success rate in black-box attacks while maintaining a high success rate in white-box attacks. This phenomenon is due to the fact that the ESMI-FGSM method combines neighborhood data point sampling algorithms and spatial momentum algorithms, considering more information when calculating gradients. This allows for the correction of the gradient update direction in the current iteration, effectively stabilizing the gradient update direction and thus maximizing the overcoming of local extrema. The method further considers the impact of intermediate layer features on the overall model decision-making, and selectively processes intermediate layer features. This helps generate more powerful adversarial examples, thereby effectively improving the success rate of black-box attacks.

[0075] This invention aims to accurately and efficiently reveal potential problems in the security protection of intelligent security systems. By observing the system's identification results of adversarial examples, it is possible to gain in-depth insight into whether there are critical issues such as misjudgments, missed alarms, or false alarms. This helps to promptly discover and fix system vulnerabilities, promoting the further improvement of intelligent security systems. This will enable the system to provide users with a safer and more reliable security protection experience, ensuring that intelligent security systems can operate stably in various complex environments.

[0076] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A black-box adversarial attack method for evaluating the robustness of intelligent security systems, characterized in that, include: S1. Obtain a dataset D similar to the training intelligent security system, use dataset D to train an alternative model, and obtain the trained alternative model. S2. Obtain the input image from dataset D, and construct the neighborhood image of the input image using the neighborhood data point sampling algorithm; S3. Preprocess the neighborhood image, input the preprocessed image into the trained substitution model, and obtain the prediction result and intermediate layer output features. S4. Integrate the spatial momentum algorithm and calculate the average gradient based on the prediction results and intermediate layer output features; S5. Calculate spatial momentum based on the average gradient, and calculate the resistance perturbation based on the spatial momentum; S6. Combine the input image with adversarial perturbations to generate adversarial examples; if the preset maximum number of iterations T is reached, the final adversarial examples are obtained. Otherwise, return to step S2 and construct the neighborhood image of the current adversarial example; S7. Use the final adversarial sample to attack the intelligent security system and verify the robustness of the intelligent security system; The fusion of spatial momentum algorithms and the calculation of average gradients based on prediction results and intermediate layer output features include: obtaining the feature momentum of the t-th iteration. According to characteristic momentum Calculate the characteristic momentum of the (t+1)th iteration using the prediction results of the (t+1)th iteration. According to characteristic momentum The loss function value is calculated using the output features of the intermediate layer in the (t+1)th iteration, and the average gradient is calculated based on the loss function value. ； in, These are the weight values ​​updated by the gradient. , The number of neighboring images, Indicates preprocessing. This represents the i-th neighborhood image of the adversarial sample generated in the t-th iteration. This represents the true label corresponding to the input image. This represents the hyperparameters of the alternative model. Represents the loss function. This represents the average gradient generated in the (t+1)th iteration, where t is the iteration number. An index for the neighboring image; Calculate the characteristic momentum of the (t+1)th iteration This includes: calculating the gradient of the k-th layer of the surrogate model in the (t+1)-th iteration based on the prediction results of the surrogate model. ,according to Calculate the characteristic momentum of the (t+1)th iteration : ； in, For weights, for Norm, Let be the characteristic momentum of the t-th iteration; The calculation of the loss function value includes: ； in, Let M represent matrix multiplication, where M is the substituted model's . The number of features output by the layer For the replacement model, the (t+1)th iteration of the... The j-th feature output by the layer.

2. The black-box adversarial attack method for evaluating the robustness of an intelligent security system according to claim 1, characterized in that, Obtaining a dataset similar to that used for training the intelligent security system involves interacting with the input and output interfaces of the intelligent security system and repeatedly querying and observing the output of the intelligent security system to obtain a dataset similar to that used for training the intelligent security system.

3. The black-box adversarial attack method for evaluating the robustness of an intelligent security system according to claim 1, characterized in that, The neighborhood image for constructing adversarial examples includes: ； in, This represents the adversarial example generated in the t-th iteration. This represents the boundary radius of the adversarial sample neighborhood sampling. Indicates in Uniform distribution in 2D space Indicates the perturbation threshold. Let represent the i-th neighborhood image of the adversarial sample generated in the t-th iteration, where t is the iteration number.

4. The black-box adversarial attack method for evaluating the robustness of an intelligent security system according to claim 1, characterized in that, Preprocessing the neighborhood image includes: dividing the neighborhood image into multiple sub-images at different locations; and applying a random left-right flip transformation to all sub-images with probability q.

5. A black-box adversarial attack method for evaluating the robustness of an intelligent security system according to claim 1, characterized in that, Calculating spatial momentum based on the average gradient includes: ； in, Let be the space momentum of the t-th iteration. Let be the space momentum of the (t+1)th iteration. The momentum decay factor, Let be the average gradient of the (t+1)th iteration, where t is the iteration number. for Norm.

6. A black-box adversarial attack method for evaluating the robustness of an intelligent security system according to claim 1, characterized in that, Generating adversarial examples by combining input images with adversarial perturbations includes: ； in, For the adversarial examples generated in the (t+1)th iteration, To counteract disturbances, For the updated step size, Let x be the input image, and x be the symbolic function. Let be the spatial momentum of the (t+1)th iteration, where t is the iteration number.

Images