Adversarial defense method, device, medium and equipment for dynamic image classification network

Abstract

The application discloses an adversarial defense method and device of a dynamic image classification network, a medium and equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring dynamic image information to be identified at a first number of time steps; extracting dynamic image information to be identified at a second number of time steps from the dynamic image information to be identified at the first number of time steps by using a randomization time step layer, wherein the second number of time steps is less than the first number of time steps; and inputting the extracted dynamic image information to be identified at the second number of time steps into a preset dynamic image classification network for classification to obtain a dynamic image classification label. The randomization time step layer is independent of the preset dynamic image classification network, and the preset dynamic image classification network comprises a pulse neural network. When the adversarial defense is performed, the network can be avoided from being retrained, so that the training cost can be reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an adversarial defense method, apparatus, medium and device for dynamic image classification networks. Background Technology

[0002] Dynamic image classification networks can be applied to scenarios such as human activity recognition, environmental monitoring during autonomous driving, and anomaly detection in surveillance. However, dynamic image classification networks are vulnerable to attacks during the inference phase. For example, adversarial examples, with just a carefully designed minor modification to the original data, can cause the spiking neural network to output incorrect results. In image classification tasks, attackers can create adversarial examples that cause the dynamic image classification network to output a category that is no longer correct.

[0003] Currently, to improve the robustness of dynamic image classification networks, adversarial training or adversarial detection is usually required. However, both adversarial training and adversarial detection require retraining the network, which leads to high training costs. Summary of the Invention

[0004] In view of this, this application provides an adversarial defense method, apparatus, medium and device for dynamic image classification networks, the main purpose of which is to avoid retraining the network when performing adversarial defense, thereby reducing training costs.

[0005] According to a first aspect of this application, an adversarial defense method for dynamic image classification networks is provided, the method comprising:

[0006] Obtain the dynamic image information to be identified at the first number of time steps;

[0007] The randomized time step layer is used to extract the dynamic image information to be identified for a second time step from the dynamic image information to be identified for a first number of time steps, wherein the second number of time steps is less than the first number of time steps.

[0008] The extracted dynamic image information of the second number of time steps is input into a preset dynamic image classification network for classification to obtain dynamic image classification labels. The randomized time step layer is independent of the preset dynamic image classification network, which includes a spiking neural network.

[0009] According to a second aspect of this application, an adversarial defense device for dynamic image classification networks is provided, the device comprising:

[0010] The acquisition unit is used to acquire the dynamic image information to be identified at a first number of time steps;

[0011] An extraction unit is used to extract a second number of time steps of dynamic image information to be identified from the first number of time steps of dynamic image information to be identified using a randomized time step layer, wherein the second number of time steps is less than the first number of time steps.

[0012] A classification unit is used to input the extracted dynamic image information of the second number of time steps to be identified into a preset dynamic image classification network for classification to obtain dynamic image classification labels. The randomized time step layer is independent of the preset dynamic image classification network, which includes a spiking neural network.

[0013] According to a third aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the adversarial defense method of the dynamic image classification network described above.

[0014] According to a fourth aspect of this application, an electronic device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the adversarial defense method of the dynamic image classification network described above.

[0015] By employing the above technical solutions, this application provides an adversarial defense method, apparatus, medium, and device for dynamic image classification networks. This method improves the robustness of the preset dynamic image classification network against adversarial attacks by using a randomized time-step layer independent of the preset dynamic image classification network to randomly extract dynamic image information to be identified from a first number of time steps, and then inputting the randomly extracted dynamic image information from a second number of time steps into the preset dynamic image classification network for classification. Since the randomized time-step layer of this application is independent of the preset dynamic image classification network and has a plug-and-play feature, it avoids retraining the network, thereby reducing training costs.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A flowchart illustrating an adversarial defense method for a dynamic image classification network provided in an embodiment of this application is shown.

[0019] Figure 2 This application illustrates dynamic image information acquired by an event-driven sensor according to an embodiment of the present application;

[0020] Figure 3 This paper illustrates a schematic diagram of the processing flow of the randomized time step layer provided in an embodiment of this application.

[0021] Figure 4 This illustration shows a schematic diagram of the extraction principle of the randomized time step layer provided in an embodiment of this application;

[0022] Figure 5 This paper presents a schematic diagram of the structure of an adversarial defense device for a dynamic image classification network provided in an embodiment of this application. Detailed Implementation

[0023] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0024] This invention provides an adversarial defense method for dynamic image classification networks. When used independently, it can defend against adversarial attacks to a certain extent without retraining the network, and it can also work in conjunction with other adversarial defense methods (including adversarial training and adversarial detection) to defend against adversarial attacks.

[0025] This invention provides an adversarial defense method for dynamic image classification networks, such as... Figure 1 As shown, the method includes:

[0026] Step 10: Obtain the dynamic image information to be recognized at the first number of time steps.

[0027] The dynamic image information to be identified refers to image data with dimensions such as time step, channel, length, and width. This dynamic image information can refer to dynamic image information in scenarios such as human activity recognition, environmental monitoring during autonomous driving, and anomaly recognition in surveillance. In the human activity recognition scenario, the dynamic image information to be identified specifically refers to human dynamic image information. The environmental monitoring scenario during autonomous driving can be further subdivided into object detection and road surface detection. In the object detection scenario, the dynamic image information to be identified specifically refers to dynamic image information containing vehicles, pedestrians, or other objects. In the road surface detection scenario, the dynamic image information to be identified specifically refers to road surface dynamic image information. In the anomaly recognition scenario in surveillance, the dynamic image information to be identified specifically refers to surveillance dynamic image information.

[0028] This invention is primarily applicable to adversarial defense scenarios for dynamic image classification networks. The executing entity of this invention is a device or equipment, such as a server, capable of performing adversarial defense against dynamic image classification networks.

[0029] In the embodiments of the present invention, under the above application scenarios, an event-driven sensor can be used to collect dynamic image information to be identified over a first number of time steps, for example, to obtain T. s The event-driven sensor includes a Dynamic Vision Sensor (DVS). DVS captures rapidly changing events in the environment and records light intensity changes asynchronously. This aligns perfectly with the event-driven computation mechanism of spiking neural networks (SNNs), allowing SNNs to leverage their unique advantages in processing DVS data. Especially in applications requiring low power consumption and real-time processing, SNNs demonstrate significant potential and can acquire data over time, exhibiting multi-input time-step characteristics. The specific acquired dynamic image information to be identified is as follows: Figure 2 As shown.

[0030] Step 20: Extract the dynamic image information to be identified for a second number of time steps from the dynamic image information to be identified for the first number of time steps using a randomized time step layer.

[0031] The second number of time steps is less than the first number of time steps. The output of the randomized time step layer is connected to the input of a preset dynamic image classification network.

[0032] In view of the characteristics of multi-time-step input data, this embodiment of the invention directly adds a randomized time-step layer before the preset dynamic image classification network, i.e., the spiking neural network. This randomized time-step layer can be directly applied to the inference stage of the preset dynamic image classification network to improve the robustness against adversarial attacks.

[0033] This invention provides a specific method for randomizing data extraction using a randomized time-step layer, such as... Figure 3 As shown, it includes:

[0034] Step 21: Set the first quantity time step and the second quantity time step.

[0035] In this embodiment of the invention, the first quantity time step is related to the time step of the sample dynamic image information and can be determined based on the time step of the sample dynamic image information. The second quantity time step can be set according to actual business requirements, and the second quantity time step is less than the first quantity time step. For example, the first quantity time step is T. s The second quantity is T. t One, T t Less than T s .

[0036] Step 22: Determine the input order of the dynamic image information to be identified at different time steps based on the dynamic image information to be identified at the first number of time steps.

[0037] In this embodiment of the invention, it is necessary to determine the input order of the dynamic image information to be identified in the first number of time steps, so as to ensure that the dynamic image information to be identified in the second number of time steps is also input in this order.

[0038] Step 23: Based on the input order, extract the dynamic image information to be identified for the second number of time steps from the dynamic image information to be identified for the first number of time steps using the randomized time step layer.

[0039] The randomized time step layer is added to the front end of the preset dynamic image classification network.

[0040] like Figure 4 As shown, the source input is Xs, which represents T. s The dynamic image information to be identified at each time step is used to extract information from T using a randomized time step layer. s Extracting dynamic image information from each time step (T) t The dynamic image information to be recognized at each time step is used as the actual input Xt. During extraction, it is necessary to ensure that the extracted T... t The input order of the dynamic image information to be recognized at each time step is consistent with the source input order.

[0041] The embodiments of the present invention can improve the accuracy of spiking neural networks under white-box adversarial attacks by introducing a randomized time step layer during the inference phase.

[0042] Step 30: Input the extracted dynamic image information of the second number of time steps to be identified into a preset dynamic image classification network for classification to obtain dynamic image classification labels.

[0043] The randomized time step layer is independent of the preset dynamic image classification network, which includes spiking neural networks (SNNs). Furthermore, in human activity recognition scenarios, the dynamic image classification label specifically refers to human behavior, including walking, running, squatting, etc.; in object detection scenarios, the dynamic image classification label specifically refers to pedestrians, vehicles, other objects, etc.; in road surface detection scenarios, the dynamic image classification label specifically refers to the lane where the vehicle is located; and in anomaly recognition scenarios during surveillance, the dynamic image classification label specifically indicates whether an anomaly exists or not.

[0044] In this embodiment of the invention, in order to determine the dynamic image classification label, step 103 specifically includes: inputting the extracted dynamic image information to be identified at the second number of time steps into the spiking neural network for encoding to obtain a pulse signal; transmitting and calculating the pulse signal between neurons in the preset dynamic image classification network to output the dynamic image classification label.

[0045] In some embodiments, the randomized timestep layer can be applied not only to the inference stage of a spiking neural network but also to its training stage. That is, a randomized timestep layer is introduced during the training stage as a data augmentation technique to enhance the network's generalization ability at different timesteps. Based on this, the method includes: collecting sample dynamic image information and its corresponding actual dynamic image classification labels; constructing an initial randomized timestep layer and an initial dynamic image classification network; inputting the sample dynamic image information into the initial randomized timestep layer for random sampling to obtain the sampled dynamic image information; inputting the extracted sample dynamic image information into the initial dynamic image classification network for classification to obtain predicted dynamic image classification labels; constructing a loss function based on the predicted dynamic image classification labels and the actual dynamic image classification labels; and iteratively training the initial randomized timestep layer and the initial dynamic image classification network based on the loss function until a preset iteration condition is met, and then outputting the randomized timestep layer and the preset dynamic image classification network.

[0046] The preset iteration conditions specifically include a preset number of iterations, meaning that training stops when the network reaches the preset number of iterations.

[0047] When attacked, if the attacker's model contains a randomized time step layer, the extracted sample dynamic image information is used as input data to calculate the gradient of the loss function with respect to the input data, and the gradient is backpropagated to the attacker.

[0048] Specifically, when facing adversarial attacks, the attacker needs to obtain the gradient of the loss function with respect to the input. If the attacker's model contains a randomized time-step layer but its existence is not detected, the gradient with respect to the input will pass through the randomized time-step layer during backpropagation, propagating the gradient to the attacker. At this time, the randomized time-step layer retains the true gradient of the time-step data extracted during the randomization process and input into the network, while setting the gradient of the time-step data filtered out during the randomization process to zero. Since the attacker can only obtain a portion of the valid gradients as the basis for the attack, this embodiment of the invention can improve the robustness of spiking neural networks against adversarial attacks.

[0049] Furthermore, the embodiments of the present invention can also verify the above method through experiments. Specifically, based on whether the attacker obtains the randomized time step layer at the network input simultaneously with acquiring the parameters of the preset dynamic image classification network, the attack experiments can be divided into Experiment 1, Experiment 2, and Experiment 3. In Experiment 1, the attacker uses a white-box attack model without a randomized time step layer, and the preset dynamic image classification network also lacks a randomized time step layer. Experiment 1 serves as a baseline for comparison, testing the basic accuracy of the preset dynamic image classification network under attack without robustness enhancement methods. In Experiment 2, the attacker uses a white-box attack model with a randomized time step layer; that is, the attacker uses adversarial examples of neural networks containing weight parameters and a randomized time step layer to attack the preset dynamic image classification network. In Experiment 3, the attacker uses a white-box attack model without a randomized time step layer; that is, the attacker uses adversarial examples of neural networks containing only weight parameters to attack the preset dynamic image classification network. Analysis of the experimental data from the three experiments reveals that the image classification accuracy of Experiments 2 and 3 is higher than that of Experiment 1. This demonstrates that inserting a randomized timestep layer at the front end of the pre-defined dynamic image classification network can improve its robustness against adversarial attacks. Furthermore, the higher image classification accuracy in Experiment 2 compared to Experiment 3 indicates that the attacker's white-box attack model, with its randomized timestep layer, achieves higher image classification accuracy.

[0050] This invention provides an adversarial defense method, apparatus, medium, and device for a dynamic image classification network. By employing a randomized time-step layer independent of the preset dynamic image classification network, the system randomly extracts dynamic image information to be identified from a first number of time steps. This randomly extracted dynamic image information from a second number of time steps is then input into the preset dynamic image classification network for classification. This improves the robustness of the preset dynamic image classification network against adversarial attacks. Because the randomized time-step layer in this invention is independent of the preset dynamic image classification network and has a plug-and-play feature, it avoids retraining the network, thereby reducing training costs.

[0051] Furthermore, as Figure 1 and Figure 3 The specific implementation of the method shown in this embodiment provides an adversarial defense device for dynamic image classification networks, such as... Figure 5 As shown, the device includes: an acquisition unit 101, an extraction unit 102, and a classification unit 103.

[0052] The acquisition unit 101 can be used to acquire the dynamic image information to be identified at a first number of time steps.

[0053] The extraction unit 102 can be used to extract a second number of time steps of dynamic image information to be identified from the first number of time steps of dynamic image information to be identified using a randomized time step layer, wherein the second number of time steps is less than the first number of time steps.

[0054] The classification unit 103 can be used to input the extracted dynamic image information of the second number of time steps to a preset dynamic image classification network for classification to obtain dynamic image classification labels. The randomized time step layer is independent of the preset dynamic image classification network, which includes a spiking neural network.

[0055] In some embodiments, the extraction unit 102 may be specifically used to set the first number of time steps and the second number of time steps; determine the input order between the dynamic image information to be identified at different time steps based on the dynamic image information to be identified at the first number of time steps; and extract the dynamic image information to be identified at the second number of time steps from the dynamic image information to be identified at the first number of time steps using the randomized time step layer based on the input order.

[0056] In some embodiments, the output of the randomized time step layer is connected to the input of the preset dynamic image classification network.

[0057] In some embodiments, the acquisition unit 101 may be specifically used to acquire the dynamic image information to be identified at the first number of time steps using an event-driven sensor.

[0058] In some embodiments, the classification unit 103 may be specifically used to input the extracted dynamic image information to be identified at the second number of time steps into the spiking neural network for encoding to obtain a pulse signal; and to transmit and calculate the pulse signal between the neurons of the preset dynamic image classification network to output a dynamic image classification label.

[0059] In some embodiments, the apparatus further includes a training unit.

[0060] The training unit can be used to collect sample dynamic image information and its corresponding actual dynamic image classification labels; construct an initial randomized time step layer and an initial dynamic image classification network; input the sample dynamic image information into the initial randomized time step layer for random sampling to obtain the sample dynamic image information; input the sample dynamic image information into the initial dynamic image classification network for classification to obtain predicted dynamic image classification labels; construct a loss function based on the predicted dynamic image classification labels and the actual dynamic image classification labels; and iteratively train the initial randomized time step layer and the initial dynamic image classification network based on the loss function until a preset iteration condition is met, and then output the randomized time step layer and the preset dynamic image classification network.

[0061] In some embodiments, the apparatus further includes a propagation unit.

[0062] The propagation unit can be used to take the extracted sample dynamic image information as input data when the attacker's model includes the randomized time step layer, calculate the gradient of the loss function with respect to the input data, and backpropagate the gradient to the attacker.

[0063] It should be noted that other corresponding descriptions of the functional units involved in the adversarial defense device for a dynamic image classification network provided in this embodiment of the invention can be found in the following references. Figure 1 and Figure 3 The corresponding description in [the document] will not be repeated here.

[0064] Based on the above, Figure 1 and Figure 3 Accordingly, this embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described method. Figure 1 and Figure 3 Adversarial defense methods for dynamic image classification networks are shown.

[0065] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause an electronic device (such as a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0066] Based on the above, Figure 1 and Figure 3 The method shown, and Figure 5To achieve the above objectives, the present application also provides an electronic device, specifically a personal computer, tablet computer, server, or other network device, as shown in the virtual device embodiment. This device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figure 1 and Figure 3 Adversarial defense methods for dynamic image classification networks are shown.

[0067] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0068] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0069] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware.

[0071] This invention improves the robustness of the preset dynamic image classification network against adversarial attacks by employing a randomized time-step layer independent of the preset dynamic image classification network. Since the randomized time-step layer is independent of the preset dynamic image classification network and has a plug-and-play feature, retraining the network can be avoided, thus reducing training costs.

[0072] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0073] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. An adversarial defense method for dynamic image classification networks, characterized in that, include: Obtain the dynamic image information to be identified at the first number of time steps; The randomized time step layer is used to extract the dynamic image information to be identified for a second time step from the dynamic image information to be identified for a first number of time steps, wherein the second number of time steps is less than the first number of time steps. The extracted dynamic image information of the second number of time steps is input into a preset dynamic image classification network for classification to obtain dynamic image classification labels. The randomized time step layer is independent of the preset dynamic image classification network, which includes a spiking neural network.

2. The method according to claim 1, characterized in that, The step of extracting a second number of time-step dynamic image information from the first number of time-step dynamic image information using a randomized time-step layer includes: Set the first quantity time step and the second quantity time step; Based on the dynamic image information to be identified at the first number of time steps, determine the input order between the dynamic image information to be identified at different time steps; Based on the input order, the randomized time step layer is used to extract the dynamic image information to be identified for the second time step from the dynamic image information to be identified for the first number of time steps.

3. The method according to claim 1, characterized in that, The output of the randomized time step layer is connected to the input of the preset dynamic image classification network.

4. The method according to claim 1, characterized in that, The acquisition of the dynamic image information to be identified at the first number of time steps includes: The dynamic image information to be identified is acquired using an event-driven sensor at the first number of time steps.

5. The method according to claim 1, characterized in that, The step of inputting the extracted dynamic image information of the second number of time steps to be identified into a preset dynamic image classification network for classification to obtain dynamic image classification labels includes: The extracted dynamic image information of the second number of time steps is input into the spiking neural network for encoding to obtain a pulse signal; The pulse signal is transmitted and calculated between neurons in the preset dynamic image classification network to output dynamic image classification labels.

6. The method according to any one of claims 1-5, characterized in that, Collect sample dynamic image information and its corresponding actual dynamic image classification labels; Construct an initial randomized timestep layer and an initial dynamic image classification network; The sample dynamic image information is input into the initial randomized time step layer for random sampling to obtain the sample dynamic image information. The extracted sample dynamic image information is input into the initial dynamic image classification network for classification to obtain the predicted dynamic image classification label; Based on the predicted dynamic image classification labels and the actual dynamic image classification labels, a loss function is constructed; Based on the loss function, the initial randomized time step layer and the initial dynamic image classification network are iteratively trained until the preset iteration conditions are met, at which point the randomized time step layer and the preset dynamic image classification network are output.

7. The method according to claim 6, characterized in that, The method further includes: When the attacker's model includes the randomized time step layer, the extracted sample dynamic image information is used as input data, the gradient of the loss function with respect to the input data is calculated, and the gradient is backpropagated to the attacker.

8. An adversarial defense device for a dynamic image classification network, characterized in that, include: The acquisition unit is used to acquire the dynamic image information to be identified at a first number of time steps; An extraction unit is used to extract a second number of time steps of dynamic image information to be identified from the first number of time steps of dynamic image information to be identified using a randomized time step layer, wherein the second number of time steps is less than the first number of time steps. A classification unit is used to input the extracted dynamic image information of the second number of time steps to be identified into a preset dynamic image classification network for classification to obtain dynamic image classification labels. The randomized time step layer is independent of the preset dynamic image classification network, which includes a spiking neural network.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images