Detection device, detection method, and recording medium
By using the LPIPS distance detection device to detect and remove adversarial examples, the problem of existing technologies being unable to effectively remove adversarial examples without modifying deep learning models is solved, thus achieving security and robustness of AI services.
Patent Information
- Application Number
- CN202180098115.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-28
- Publication Date
- 2026-02-24
- Estimated Expiration
- 2041-05-28
AI Technical Summary
The lack of effective methods in existing technologies to detect and remove adversarial examples without modifying deep learning models poses a threat to the security of AI services.
The LPIPS distance detection device is used to classify adversarial samples into clean samples or adversarial samples by calculating the similarity distance between the data and the benchmark data. This enables the detection and removal of adversarial samples.
Without modifying the deep learning model, it can detect and remove adversarial examples with high accuracy, ensuring the security of AI services, preventing misclassification, and improving the robustness of the system.
Smart Images
Figure CN117280357B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a detection device, a detection method, and a recording medium. Background Technology
[0002] Adversarial examples are known to exist, which are hostile samples created by deliberately adding small perturbations to the data input to a deep learning model to distort the output (see Non-Patent Document 1). For example, in the case of adversarial examples of images, there is a problem of misclassifying the deep learning output by changing the appearance of the image without human recognition. This poses a threat, for example, considering the possibility that the type of an identifier recognized by an autonomous vehicle could be changed to another type. Thus, adversarial examples pose a significant threat to the security of AI services, and therefore, countermeasures against them are expected to be developed in deep learning.
[0003] Countermeasures against adversarial examples are categorized into several types based on their approach. For example, there are four representative approaches shown below: (1) Training a robust deep learning model that can classify adversarial examples correctly. (2) Detecting and removing adversarial examples before inputting them into the deep learning model. (3) Removing perturbations added to adversarial examples and restoring the original data. (4) Preventing the use of information from the model needed to generate adversarial examples.
[0004] Existing technical documents
[0005] Non-patent literature
[0006] Non-patent literature 1: Ian J. Goodfellow et al., “Explanating and Harnessing Adversarial Examples”, [online], March 2015, [retrieved April 26, 2021], website <URL: http: / / https: / / arxiv.org / abs / 1412.6572> Summary of the Invention
[0007] The problem that the invention aims to solve
[0008] However, in the existing technology, there is no perfect technique to prevent adversarial examples. On the other hand, since countermeasures can be taken without modifying the deep learning model, the technique described above (2) for detecting and removing adversarial examples before they are input into the deep learning model has attracted attention.
[0009] The present invention was made in view of the above circumstances, and its purpose is to detect and remove adversarial examples before they are input into a deep learning model.
[0010] Methods for solving problems
[0011] To solve the above-mentioned problems and achieve the objective, the detection apparatus of the present invention is characterized by comprising: an acquisition unit that acquires data as the detection object and normal reference data; a calculation unit that calculates the LPIPS (Learned Perceptual Image Patch Similarity) distance between the acquired data and the reference data; and a classification unit that uses the calculated LPIPS distance to classify the acquired data into clean samples or adversarial examples.
[0012] The effects of the invention
[0013] According to the present invention, adversarial examples can be detected and removed before being input into a deep learning model. Attached Figure Description
[0014] Figure 1 This is a diagram illustrating the general outline of the detection device in this embodiment.
[0015] Figure 2 This is a diagram illustrating the general outline of the detection device in this embodiment.
[0016] Figure 3 This is a schematic diagram illustrating the general structure of the detection device according to this embodiment.
[0017] Figure 4 This is a flowchart illustrating the detection and processing steps.
[0018] Figure 5 This is a diagram used to illustrate an embodiment.
[0019] Figure 6 This is a diagram used to illustrate an embodiment.
[0020] Figure 7 This is a diagram showing an example of a computer executing a detection program. Detailed Implementation
[0021] Hereinafter, one embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to this embodiment. Furthermore, in the accompanying drawings, the same reference numerals are used to denote the same parts.
[0022] [Overview of the detection device]
[0023] Figure 1 and Figure 2This is a diagram illustrating the general outline of the detection apparatus of this embodiment. The detection apparatus of this embodiment uses LPIPS (Learned Perceptual Image PatchSimilarity) distance, a state-of-the-art metric for measuring distances between images, to detect adversarial examples.
[0024] Here, the LPIPS distance is an indicator that measures the distance between images by normalizing the intermediate output of the deep learning model for each channel and taking the difference. It is calculated according to the following formula (1).
[0025] [Mathematical Expression 1]
[0026]
[0027] Here, g l (x): Output of layer l
[0028] g k (x): Output of channel c in layer l
[0029] w l Vertical dimension of the output of layer l
[0030] H l :L layer output horizontal dimension
[0031] In addition, such as Figure 1 As shown, the LPIPS distances between known adversarial examples and clean samples (which are considered normal data) and the benchmark data are different. The benchmark data is the normal data used for comparison with adversarial examples and clean samples.
[0032] Therefore, the detection device classifies adversarial examples and clean examples based on their LPIPS distance, thereby detecting adversarial examples.
[0033] Specifically, such as Figure 2 As shown, the detection device accepts inputs of data x as the detection object and its predicted category y_pred, a deep learning model g, and data belonging to category y_pred from the baseline data of normal data used for comparison. Figure 2 (1) and (1)' in the middle. In addition, the detection device calculates the LPIPS distance between the data x and the benchmark data belonging to the category y_pred ( Figure 2 (2) of the above, and output the minimum value as the anomaly score. Figure 2 (3)). Then, the detection device uses anomaly scores to classify data x as normal or abnormal, i.e., as clean samples or adversarial samples.
[0034] Therefore, the detection device can detect and remove adversarial examples before they are input into the deep learning model g, thus enabling countermeasures against adversarial examples without modifying the deep learning model g.
[0035] [Structure of the detection device]
[0036] Figure 3 This is a schematic diagram illustrating the general structure of the detection device according to this embodiment. Figure 3 As illustrated, the detection device 10 of this embodiment is implemented by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.
[0037] The input unit 11 is implemented using input devices such as a keyboard and mouse, and inputs various instruction information, such as processing start, to the control unit 15 in accordance with the operator's input operations. The output unit 12 is implemented using display devices such as liquid crystal displays, printing devices such as printers, etc. For example, the results of the detection processing described later are displayed on the output unit 12.
[0038] The communication control unit 13, implemented by a NIC (Network Interface Card) or similar device, controls communication between external devices and the control unit 15 via electrical communication lines such as LAN (Local Area Network) or the Internet. For example, the communication control unit 13 manages communication between the control unit 15 and management devices that manage data that is the object of detection processing.
[0039] The storage unit 14 is implemented using semiconductor storage elements such as RAM (Random Access Memory) and flash memory, or storage devices such as hard disks and optical disks. The storage unit 14 pre-stores the processing program that causes the detection device 10 to operate, data used when executing the processing program, etc., or temporarily stores the processing program and data each time processing is executed. In this embodiment, the storage unit 14 stores the model 14a used in the detection processing described later. Furthermore, the storage unit 14 may also be configured to communicate with the control unit 15 via the communication control unit 13.
[0040] The control unit 15 is implemented using a CPU (Central Processing Unit) or similar device, and executes processing programs stored in memory. Therefore, as... Figure 2As illustrated, the control unit 15 functions as an acquisition unit 15a, a calculation unit 15b, a training unit 15c, and a classification unit 15d. Furthermore, these functional units can be installed separately or in part on different hardware. For example, the training unit 15c can be installed on hardware different from the other functional units. Additionally, the control unit 15 may also include other functional units.
[0041] The acquisition unit 15a acquires data that is the object of detection and normal reference data. For example, the acquisition unit 15a acquires data that is the object of the detection process described later and reference data that is normal data for comparison from a management device or the like via the input unit 11 or the communication control unit 13. The acquisition unit 15a may also store the acquired data and reference data in the storage unit 14. In this case, the calculation unit 15b described later acquires the data and reference data from the storage unit 14 and performs processing.
[0042] The calculation unit 15b calculates the LPIPS distance between the acquired data and the reference data. Specifically, as described above, the calculation unit 15b calculates the LPIPS distance between the data x, which is the detection target, and the reference data, which is the normal data used for comparison, according to equation (1). In addition, the calculation unit 15b transmits the minimum value among the LPIPS distances calculated between each data and multiple reference data as the anomaly score of that data to the training unit 15c and the classification unit 15d, which will be described later.
[0043] Training unit 15c uses the calculated LPIPS distance as the anomaly score to train model 14a, which classifies data as normal or abnormal. Specifically, training unit 15c trains the model such that the anomaly scores, i.e., LPIPS distances, calculated by computation unit 15b for each data point follow a specific order. Figure 1 The illustrated distribution is used to generate a model 14a that classifies the data through training. The generated model 14a outputs whether the input data is normal (i.e., clean samples) or abnormal (i.e., adversarial samples).
[0044] The classification unit 15d uses the calculated LPIPS distance to classify the acquired data as clean samples or adversarial samples. Specifically, the classification unit 15d uses the calculated LPIPS distance as anomaly score to classify the data as normal or anomalous by model 14a, thus classifying the acquired data as normal clean samples or anomalous adversarial samples. That is, the classification unit 15d uses the trained model 14a to classify the acquired data that is the object of detection as normal (i.e., clean samples) or anomalous (i.e., adversarial samples).
[0045] In addition, if an adversarial sample is detected, the classification unit 15d outputs the adversarial sample to the output unit 12.
[0046] Therefore, the detection device 10 is able to detect adversarial examples. Thus, adversarial examples can be removed before being input into the deep learning model g, thereby enabling countermeasures against adversarial examples without modifying the deep learning model g.
[0047] [Detection and Processing]
[0048] Next, refer to Figure 4 The detection process of the detection device 10 in this embodiment will be described. Figure 4 This is a flowchart illustrating the detection and processing steps. Figure 4 The process, for example, begins at a timed start when the user has entered an instruction to begin.
[0049] First, the acquisition unit 15a acquires the data as the object of detection and the reference data (step S1). For example, the acquisition unit 15a acquires the data as the object of detection processing and the reference data as normal data for comparison via the input unit 11 or the communication control unit 13.
[0050] Next, the calculation unit 15b calculates the LPIPS distance between the data being detected and the reference data (step S2). Furthermore, the calculation unit 15b takes the minimum LPIPS distance calculated between each data point and multiple reference data points as the anomaly score for that data and passes it to the classification unit 15d.
[0051] Furthermore, the calculation unit 15b can also pass the calculated anomaly score to the training unit 15c. In this case, the training unit 15c trains the model 14a that classifies the data as normal or abnormal, and stores the generated model 14a in the storage unit 14.
[0052] Then, the classification unit 15d uses the calculated LPIPS distance to classify the acquired data as clean samples or adversarial samples (step S3). Specifically, the classification unit 15d uses the trained model 14a to classify the acquired data as detection objects as normal (i.e., clean samples) or abnormal (i.e., adversarial samples).
[0053] Furthermore, if an adversarial sample is detected, the classification unit 15d outputs the adversarial sample as a detection result to other devices via the output unit 12 and the communication control unit 13 (step S4). Thus, the series of detection processes ends.
[0054] [Effect]
[0055] As explained above, in the detection apparatus 10 of this embodiment, the acquisition unit 15a acquires data as the detection target and normal reference data. Furthermore, the calculation unit 15b calculates the LPIPS distance between the acquired data and the reference data. Additionally, the classification unit 15d uses the calculated LPIPS distance to classify the acquired data into clean samples or adversarial samples.
[0056] Specifically, the classification unit 15d uses the calculated LPIPS distance as anomaly score to classify the data as normal or anomalous according to model 14a, classifying the obtained data as normal clean samples or anomalous adversarial samples.
[0057] Therefore, the detection device 10 is capable of detecting adversarial examples. Thus, adversarial examples can be removed before being input into the deep learning model, allowing countermeasures against adversarial examples that pose a significant threat to the security of AI services without modifying the deep learning model. In this way, the security of AI services can be ensured by the detection device 10.
[0058] Furthermore, the training unit 15c uses the calculated LPIPS distance as anomaly score to train the model 14a, which classifies data as normal or anomalous. Thus, the detection device 10 can detect adversarial examples that change continuously each day with high accuracy.
[0059] [Example]
[0060] Figure 5 and Figure 6 This is a diagram used to illustrate an embodiment. Figure 5 It shows that it is aimed at Figure 1 The histogram shows the ROC (Receiver Operating Characteristic Curve) of the detection results of the above-described implementation of the adversarial and clean samples. Additionally, in this embodiment, the deep learning model used is ResNet18, the dataset used is CIFAR10, and the adversarial samples used are PGD (see "https: / / arxiv.org / abs / 1706.06083").
[0061] As a result, in Figure 5 In the example shown, the AUC (Area Under Curve) value of the ROC curve, which indicates that the closer to model 1, the higher the performance, is 0.88, confirming that adversarial examples can be detected with high accuracy.
[0062] Furthermore, in Figure 6 The application of the detection device 10 of the above-described embodiment in a labeling and classification system is illustrated. Figure 6In the illustrated sign classification system, within the detection device 10, the acquisition unit 15a acquires signs captured by a camera as detection target data, and the classification unit 15d detects adversarial examples. Furthermore, the detection device 10 discards an adversarial example if it detects one, and inputs the sample into a deep learning model if no adversarial example is detected. This, for example, can prevent autonomous vehicles from misidentifying signs, ensuring the safety of vehicle control.
[0063] [program]
[0064] It is also possible to generate a program in a computer-executable language that describes the processing performed by the detection device 10 according to the above-described embodiments. As one embodiment, the detection device 10 can be implemented by installing the detection program for performing the above-described detection processing as a packaged software or online software on a desired computer. For example, by having an information processing device execute the above-described detection program, the information processing device can function as the detection device 10. The information processing device referred to here includes desktop or laptop personal computers. In addition, information processing devices also include smartphones, mobile phones, mobile communication terminals such as PHS (Personal Handyphone System), and tablet terminals such as PDA (Personal Digital Assistant). Furthermore, the functions of the detection device 10 can also be implemented on a cloud server.
[0065] Figure 7 This diagram illustrates an example of a computer executing a testing program. The computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected via a bus 1080.
[0066] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, boot programs such as the BIOS (Basic Input Output System). A hard disk drive interface 1030 is connected to a hard disk drive 1031. A disk drive interface 1040 is connected to a disk drive 1041. Removable storage media such as disks or optical discs are inserted into the disk drive 1041. A serial port interface 1050 is connected, for example, to a mouse 1051 and a keyboard 1052. A video adapter 1060 is connected, for example, to a monitor 1061.
[0067] Here, the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. The information described in the above embodiments is stored, for example, in the hard disk drive 1031 and the memory 1010.
[0068] Additionally, the detection program is stored in the hard disk drive 1031, for example, as a program module 1093 containing instructions executed by the computer 1000. Specifically, the program module 1093 containing the various processes executed by the detection device 10 as described in the above embodiment is stored in the hard disk drive 1031.
[0069] Additionally, the data used in the information processing based on the detection program is stored as program data 1094, for example, in the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as needed, thereby executing the steps described above.
[0070] Furthermore, the program module 1093 and program data 1094 involved in the detection program are not limited to being stored on the hard disk drive 1031. For example, they can also be stored on a removable storage medium and read by the CPU 1020 via the disk drive 1041, etc. Alternatively, the program module 1093 and program data 1094 involved in the detection program can also be stored on other computers connected via a network such as a LAN or WAN (Wide Area Network) and read by the CPU 1020 via the network interface 1070.
[0071] The embodiments of the invention made by the inventors have been described above, but the present invention is not limited to the description and drawings that constitute a part of the disclosure of the present invention. That is, all other embodiments, examples, and techniques applied based on the present embodiments by those skilled in the art are included within the scope of the present invention.
[0072] Label Explanation
[0073] 10 Detection Device
[0074] 11 Input Section
[0075] 12 Output Section
[0076] 13. Communications Control Department
[0077] 14 Storage Department
[0078] Model 14a
[0079] 15. Control Department
[0080] 15a Acquisition Department
[0081] 15b Computing Department
[0082] 15c Training Department
[0083] 15d Classification Department
Claims
1. A detection device, characterized in that, have: The acquisition department acquires data as the object of testing and normal baseline data; The computing unit calculates the LPIPS distance, i.e., the learning-aware image patch similarity distance, between the obtained data and the reference data. as well as The classification department, based on the distribution of the different LPIPS distances in clean samples and adversarial samples, uses the calculated LPIPS distances to classify the acquired data as either clean samples or adversarial samples. The classification department uses a model to classify the acquired data as normal clean samples or anomalous adversarial samples, wherein the model uses the minimum of the LPIPS distances calculated between the data and a plurality of the benchmark data as the anomalous score to classify the data as normal or anomalous.
2. The detection device according to claim 1, characterized in that, The detection device also includes a training unit, which uses the calculated LPIPS distance as an anomaly score to train the model that classifies data as normal or abnormal.
3. The detection device according to claim 1, characterized in that, The acquisition unit acquires the identifier captured by the camera as the detection object data. The classification department detects adversarial samples.
4. A detection method, wherein the detection method is performed by a detection device, characterized in that, The detection method includes: The process involves obtaining data as the object of inspection and normal baseline data. The calculation process includes calculating the LPIPS distance, i.e., the learned perceptual image patch similarity distance, between the acquired data and the benchmark data; and The classification process involves classifying the acquired data into clean samples or adversarial samples based on the different distributions of the LPIPS distances in clean samples and adversarial samples, using the calculated LPIPS distances. In the classification process, the acquired data is classified into normal clean samples or anomalous adversarial samples using a model that uses the minimum of the LPIPS distances calculated between the data and multiple benchmark data as the anomalous score to classify the data as normal or anomalous.
5. A recording medium storing a detection program for causing a computer to perform the following steps: The acquisition steps involve obtaining data as the object of detection and normal baseline data; The calculation steps include calculating the LPIPS distance, i.e., the learned perceptual image patch similarity distance, between the obtained data and the benchmark data; and The classification step involves classifying the acquired data into clean samples or adversarial samples based on the different distributions of the LPIPS distances in clean and adversarial samples, using the calculated LPIPS distances. In the classification step, the acquired data is classified as normal clean samples or anomalous adversarial samples using a model that uses the minimum of the LPIPS distances calculated between the data and multiple benchmark data as the anomalous score to classify the data as normal or anomalous.