DETECTION APPARATUS, DETECTION METHOD, AND DETECTION PROGRAM

The detection device employs LPIPS distance to identify and eliminate adversarial examples, addressing the threat to AI safety by pre-emptive classification and removal, ensuring robustness without altering the deep learning model.

JP7593491B2Active Publication Date: 2024-12-03NIPPON TELEGRAPH & TELEPHONE CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023523928
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-05-28
Publication Date
2024-12-03
Estimated Expiration
2041-05-28

AI Technical Summary

Technical Problem

Existing technologies lack a comprehensive method to effectively detect and remove adversarial examples before they are input into deep learning models, posing a significant threat to the safety of AI services.

Method used

A detection device that utilizes the Learned Perceptual Image Patch Similarity (LPIPS) distance to classify data as either clean samples or adversarial examples, allowing for their detection and removal before input into deep learning models.

Benefits of technology

The solution enables high-accuracy detection and removal of adversarial examples, ensuring the safety of AI services without modifying the deep learning model, thereby enhancing the robustness of AI systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007593491000002
    Figure 0007593491000002
  • Figure 0007593491000003
    Figure 0007593491000003
  • Figure 0007593491000004
    Figure 0007593491000004
Patent Text Reader

Abstract

An acquisition unit (15a) acquires detection target data and normal reference data. A calculation unit (15b) calculates the Learned Perceptual Image Patch Similarity (LPIPS) distance between the acquired detection target data and normal reference data. A classification unit (15d) obtains data which is classified using a model that uses the calculated LPIPS distance to acquire data to classify the aforementioned acquired data as either a Clean Sample or an Adversarial Example.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a detection device, a detection method, and a detection program. [Background technology]

[0002] The existence of adversarial examples, which are adversarial samples created by intentionally adding minute perturbations to data input to a deep learning model so as to confuse the output, is known (see Non-Patent Document 1). For example, there is a problem that adversarial examples of images misclassify the output of deep learning while leaving the appearance unchanged and unrecognizable to humans. For example, there is a threat that they may change the type of sign recognized by an autonomous vehicle to a different one. As such, adversarial examples pose a major threat to the safety of AI services, and therefore there are hopes for the creation of countermeasure technologies in deep learning.

[0003] Countermeasures against adversarial examples are classified into several categories depending on the approach. For example, there are four typical approaches: (1) Train a robust deep learning model that can correctly classify adversarial examples. (2) Detect and remove adversarial examples before inputting them into the deep learning model. (3) Remove the perturbations added to the adversarial examples to restore the original data. (4) Prevent the use of information from the model required when creating adversarial examples. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Ian J.Goodfellow et al., “EXPLAINING AND HARNESSING ADVERSARIAL EXAMPLES”, [online], March 2015, [Retrieved April 26, 2021], Internet<URL:http: / / https: / / arxiv.org / abs / 1412.6572> Summary of the Invention [Problem to be solved by the invention]

[0005] However, there is no conventional technology that can completely prevent adversarial examples. On the other hand, the technology mentioned in (2) above that detects and removes adversarial examples before they are input into a deep learning model is attracting attention because it is possible to take measures without modifying the deep learning model.

[0006] The present invention has been made in consideration of the above, and aims to detect and remove adversarial examples before inputting them into a deep learning model. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the objective, the detection device of the present invention is characterized by having an acquisition unit that acquires data of a detection target and normal reference data, a calculation unit that calculates an LPIPS (Learned Perceptual Image Patch Similarity) distance between the acquired data and the reference data, and a classification unit that classifies the acquired data into either a Clean Sample or an Adversarial Example using the calculated LPIPS distance. Effect of the Invention

[0008] According to the present invention, it is possible to detect and remove adversarial examples before inputting them into a deep learning model. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining an overview of the detection device of the present embodiment. [Diagram 2] FIG. 2 is a diagram for explaining an outline of the detection device of this embodiment. [Diagram 3] FIG. 3 is a schematic diagram illustrating a schematic configuration of the detection device of the present embodiment. [Figure 4] FIG. 4 is a flowchart showing the procedure of the detection process. [Diagram 5] FIG. 5 is a diagram for explaining an embodiment. [Figure 6] FIG. 6 is a diagram for explaining an embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a computer that executes a detection program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are indicated by the same reference numerals.

[0011] [Detection device overview] 1 and 2 are diagrams for explaining an overview of the detection device of this embodiment. The detection device of this embodiment detects Adversarial Examples using the Learned Perceptual Image Patch Similarity (LPIPS) distance, which is the latest index for measuring the distance between images.

[0012] Here, the LPIPS distance is an index that measures the distance between images by normalizing the intermediate output of the deep learning model for each channel and taking the difference, and is a value calculated as shown in the following equation (1).

[0013]

number

[0014] In addition, as shown in FIG. 1, it is known that the LPIPS distance between an Adversarial Example and a Clean Sample, which is normal data, differs from that between these samples and reference data, which is normal data for comparison.

[0015] Therefore, the detection device detects Adversarial Examples by classifying Adversarial Examples and Clean Samples based on the difference in LPIPS distance.

[0016] Specifically, as shown in FIG. 2, the detection device receives input of data x to be detected, its predicted class y_pred, a deep learning model g, and reference data that is normal data for comparison, among which data belongs to class y_pred ((1) and (1)' in FIG. 2). The detection device also calculates the LPIPS distance between the data x and the reference data that belongs to class y_pred ((2) in FIG. 2), and outputs the minimum value of the distance as the anomaly score ((3) in FIG. 2). The detection device then uses the anomaly score to classify the data x as normal / abnormal, i.e., Clean Sample / Adversarial Example.

[0017] This enables the detection device to detect and remove Adversarial Examples before inputting them into the deep learning model g, making it possible to take measures against Adversarial Examples without modifying the deep learning model g.

[0018] [Detection device configuration] Fig. 3 is a schematic diagram illustrating the schematic configuration of the detection device of this embodiment. As illustrated in Fig. 3, the detection device 10 of this embodiment is realized by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

[0019] The input unit 11 is realized by using input devices such as a keyboard and a mouse, and inputs various instruction information such as a command to start processing to the control unit 15 in response to an input operation by an operator. The output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, etc. For example, the output unit 12 displays the results of a detection process described below.

[0020] The communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between the control unit 15 and an external device via a telecommunication line such as a LAN (Local Area Network) or the Internet. For example, the communication control unit 13 controls communication between the control unit 15 and a management device or the like that manages data to be subjected to detection processing.

[0021] The storage unit 14 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 14 stores in advance a processing program for operating the detection device 10, data used during execution of the processing program, and the like, or temporarily stores the data each time processing is performed. In this embodiment, the storage unit 14 stores a model 14a used in the detection process described below. The storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.

[0022] The control unit 15 is realized using a CPU (Central Processing Unit) or the like, and executes a processing program stored in a memory. As a result, the control unit 15 functions as an acquisition unit 15a, a calculation unit 15b, a learning unit 15c, and a classification unit 15d, as exemplified in FIG. 2. Note that each or some of these functional units may be implemented in different hardware. For example, the learning unit 15c may be implemented in hardware different from the other functional units. The control unit 15 may also include other functional units.

[0023] The acquisition unit 15a acquires data to be detected and normal reference data. For example, the acquisition unit 15a acquires data to be detected in a detection process (to be described later) and reference data, which is normal data for comparison, from a management device or the like via the input unit 11 or the communication control unit 13. The acquisition unit 15a may store the acquired data and reference data in the storage unit 14. In this case, the calculation unit 15b (to be described later) acquires the data and reference data from the storage unit 14 and performs processing.

[0024] The calculation unit 15b calculates the LPIPS distance between the acquired data and the reference data. Specifically, the calculation unit 15b calculates the LPIPS distance between the data x to be detected and the reference data, which is normal data for comparison, according to the formula (1) as described above. In addition, the calculation unit 15b passes the minimum value of the LPIPS distances calculated between each data and the multiple reference data as the abnormality score of the data to the learning unit 15c and the classification unit 15d described later.

[0025] The learning unit 15c learns the model 14a that classifies data into either normal or abnormal using the calculated LPIPS distance as an anomaly score. Specifically, the learning unit 15c generates the model 14a that classifies each data by learning so that the anomaly score of each data, i.e., the LPIPS distance, calculated by the calculation unit 15b follows the distribution illustrated in Fig. 1. The model 14a generated here outputs whether the input data is normal, i.e., a clean sample, or abnormal, i.e., an adversarial example.

[0026] The classification unit 15d classifies the acquired data into either a Clean Sample or an Adversarial Example using the calculated LPIPS distance. Specifically, the classification unit 15d classifies the acquired data into either a normal Clean Sample or an abnormal Adversarial Example using the model 14a that classifies data into either normal or abnormal using the calculated LPIPS distance as an abnormality score. In other words, the classification unit 15d classifies the acquired data of the detection target into either normal, i.e., a Clean Sample, or abnormal, i.e., an Adversarial Example, using the learned model 14a.

[0027] Furthermore, when the classification unit 15d detects an Adversarial Example, it outputs the Adversarial Example to the output unit 12.

[0028] This enables the detection device 10 to detect Adversarial Examples. Therefore, it is possible to remove Adversarial Examples before inputting them to the deep learning model g, and to take measures against Adversarial Examples without modifying the deep learning model g.

[0029] [Detection process] Next, the detection process by the detection device 10 according to the present embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the procedure of the detection process. The flowchart in Fig. 4 starts, for example, when a user performs an operation input to instruct the start of the process.

[0030] First, the acquisition unit 15a acquires data to be detected and reference data (step S1). For example, the acquisition unit 15a acquires data to be detected and reference data, which is normal data for comparison, via the input unit 11 or the communication control unit 13.

[0031] Next, the calculation unit 15b calculates the LPIPS distance between the acquired detection target data and the reference data (step S2). The calculation unit 15b also transfers the minimum value of the LPIPS distances calculated between each data and the multiple reference data to the classification unit 15d as the anomaly score of the data.

[0032] The calculation unit 15b may deliver the calculated anomaly score to the learning unit 15c. In this case, the learning unit 15c learns the model 14a that classifies data into either normal or abnormal, and stores the generated model 14a in the storage unit 14.

[0033] Then, the classification unit 15d classifies the acquired data into either a Clean Sample or an Adversarial Example using the calculated LPIPS distance (step S3). Specifically, the classification unit 15d classifies the acquired data of the detection target into either normal, i.e., a Clean Sample, or abnormal, i.e., an Adversarial Example, using the trained model 14a.

[0034] Furthermore, when the classification unit 15d detects an Adversarial Example, it outputs the Adversarial Example as a detection result to another device or the like via the output unit 12 and the communication control unit 13 (step S4). This completes the series of detection processes.

[0035] [effect] As described above, in the detection device 10 of this embodiment, the acquisition unit 15a acquires data of a detection target and normal reference data. The calculation unit 15b calculates the LPIPS distance between the acquired data and the reference data. The classification unit 15d classifies the acquired data into either a Clean Sample or an Adversarial Example using the calculated LPIPS distance.

[0036] Specifically, the classification unit 15d classifies the acquired data into either a normal Clean Sample or an abnormal Adversarial Example using the calculated LPIPS distance as an abnormality score and the model 14a that classifies data into either normal or abnormal.

[0037] This enables the detection device 10 to detect Adversarial Examples. Therefore, by removing Adversarial Examples before inputting them into a deep learning model, it becomes possible to take measures against Adversarial Examples, which are a serious threat to the safety of AI services, without modifying the deep learning model. In this way, the detection device 10 makes it possible to ensure the safety of AI services.

[0038] Furthermore, the learning unit 15c learns the model 14a that classifies data into either normal or abnormal using the calculated LPIPS distance as an anomaly score, which enables the detection device 10 to accurately detect Adversarial Examples that change day by day.

[0039] [Example] 5 and 6 are diagrams for explaining the embodiment. FIG. 5 is a receiver operating characteristic curve (ROC) curve showing the result of the detection process of the above embodiment for the adversarial example and clean sample shown in the histogram of FIG. 1. In this embodiment, the deep learning model is ResNet18, the dataset is cifar10, and the adversarial example is PGD (see "https: / / arxiv.org / abs / 1706.06083").

[0040] As a result, in the example shown in Figure 5, the AUC (Area Under Curve) value of the ROC curve, which indicates that the closer to 1 the model is, the higher its performance, was 0.88, confirming that Adversarial Examples can be detected with high accuracy.

[0041] 6 illustrates an example of application of the detection device 10 of the embodiment described above to a sign classification system. In the sign classification system illustrated in FIG. 6, in the detection device 10, the acquisition unit 15a acquires a sign photographed by a camera as data of a detection target, and the classification unit 15d detects an adversarial example. If the detection device 10 detects an adversarial example, it discards it, and if it does not detect it, it inputs it into a deep learning model. This makes it possible to prevent, for example, an autonomous vehicle from erroneously recognizing a sign, thereby ensuring the safety of vehicle control.

[0042] [program] A program in which the process executed by the detection device 10 according to the above embodiment is written in a language executable by a computer can also be created. As an embodiment, the detection device 10 can be implemented by installing a detection program that executes the above detection process as package software or online software on a desired computer. For example, the above detection program can be executed by an information processing device, so that the information processing device can function as the detection device 10. The information processing device here includes desktop or notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone System), and even slate terminals such as PDA (Personal Digital Assistant). The function of the detection device 10 may be implemented on a cloud server.

[0043] 7 is a diagram showing an example of a computer that executes a detection program. The computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0044] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1031. The disk drive interface 1040 is connected to a disk drive 1041. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to a display 1061, for example.

[0045] Here, the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each piece of information described in the above embodiments is stored in the hard disk drive 1031 or memory 1010, for example.

[0046] The detection program is stored in the hard disk drive 1031 as, for example, a program module 1093 in which instructions to be executed by the computer 1000 are written. Specifically, the hard disk drive 1031 stores the program module 1093 in which each process executed by the detection device 10 described in the above embodiment is written.

[0047] Furthermore, data used for information processing by the detection program is stored as program data 1094, for example, in the hard disk drive 1031. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as necessary, and executes each of the above-mentioned procedures.

[0048] Note that the program module 1093 and program data 1094 related to the detection program are not limited to being stored in the hard disk drive 1031, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. Alternatively, the program module 1093 and program data 1094 related to the detection program may be stored in another computer connected via a network such as a LAN or a WAN (Wide Area Network), and read by the CPU 1020 via the network interface 1070.

[0049] Although the embodiment to which the invention made by the present inventor is applied has been described above, the present invention is not limited by the description and drawings which form a part of the disclosure of the present invention according to the present embodiment. In other words, other embodiments, examples, operation techniques, etc. made by those skilled in the art based on the present embodiment are all included in the scope of the present invention. [Explanation of symbols]

[0050] 10. Detection Device 11 Input section 12 Output section 13 Communication control section 14 Storage section 14a model 15 Control section 15a Acquisition part 15b Calculation part 15c Learning Section 15d Classification section

Claims

1. an acquisition unit that acquires data of a detection target and normal reference data; A calculation unit that calculates a Learned Perceptual Image Patch Similarity (LPIPS) distance between the acquired data and the reference data; a classification unit that classifies the acquired data into either a clean sample or an adverse example using the calculated LPIPS distance according to a distribution of LPIPS distances that differ between a clean sample and an adverse example; having The classification unit classifies the acquired data into either a normal Clean Sample or an abnormal Adversarial Example using a model that classifies data into either normal or abnormal using the smallest LPIPS distance calculated between the data and a plurality of the reference data as an abnormality score.

2. The detection device according to claim 1 , further comprising a learning unit configured to learn the model that classifies data as either normal or abnormal using the calculated LPIPS distance as an anomaly score.

3. The acquisition unit acquires a sign photographed by a camera as data of a detection target, The detection device according to claim 1 , wherein the classifier detects an Adversarial Example.

4. A detection method performed by a detection device, comprising: an acquisition step of acquiring data of a detection target and normal reference data; A calculation step of calculating a Learned Perceptual Image Patch Similarity (LPIPS) distance between the acquired data and the reference data; a classification step of classifying the acquired data into either a clean sample or an adverse example using the calculated LPIPS distance according to a distribution of LPIPS distances that differ between a clean sample and an adverse example; Including, The classification step is characterized by classifying the acquired data into either a normal Clean Sample or an abnormal Adversarial Example using a model that classifies data into either normal or abnormal using the smallest LPIPS distance calculated between the data and a plurality of the reference data as an abnormality score.

5. An acquisition step of acquiring data of a detection target and normal reference data; A calculation step of calculating a Learned Perceptual Image Patch Similarity (LPIPS) distance between the acquired data and the reference data; a classification step of classifying the acquired data into either a clean sample or an adverse example using the calculated LPIPS distance according to a distribution of LPIPS distances that differ between a clean sample and an adverse example; on the computer, The classification step classifies the acquired data into either a normal Clean Sample or an abnormal Adversarial Example using a model that classifies data into either normal or abnormal using the smallest LPIPS distance calculated between the data and a plurality of the reference data as an abnormality score.

Citation Information

Patent Citations

  • Detecting adversarial samples by vision based perception system

    JP2020200033A