Attack detection device, adversarial sample patch detection system, attack detection method, and computer-readable recording medium

Abstract

The attack detection device (120) has an anomaly pattern detection section (123) that detects whether an anomaly pattern is included in time series recognition scores, wherein the time series recognition scores are time series data generated using a plurality of recognition scores, the plurality of recognition scores are calculated using a plurality of image data obtained by photographing a range within a photographing range at mutually different times within a photographing time range, and represent results of detecting objects in the plurality of image data, and the anomaly pattern is generated in a case where an adversarial sample patch attack is implemented on at least any one of the plurality of image data.

Description

Technical Field

[0001] This invention relates to an attack detection device, an adversarial sample patch detection system, an attack detection method, and a computer-readable recording medium. Background Technology

[0002] In object detection tasks that use bounding boxes to represent the locations of objects in an input image and label the types of objects, deep learning methods using neural networks have achieved very high accuracy in recent years. Non-Patent Document 1 discloses a method for adversarial sample patch attacks, which physically configures adversarial sample patches printed with images subjected to electronic perturbations, thus avoiding object detection based on object detection when an image obtained by capturing the configured adversarial sample patch is input.

[0003] Existing technical documents

[0004] Non-patent literature

[0005] Non-patent document 1: S.Thys et al., "Fooling automated surveillance cameras: adversarial patches to attack person detection", CVPRW (Conference on ComputerVision and Pattern Recognition Workshops), IEEE (Institute of Electrical and Electronics Engineers) / CVF (Computer Vision Foundation), 2019 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] According to existing technology, there is a challenge in detecting adversarial sample patch attacks.

[0008] The purpose of this invention is to detect the attack when it has been subjected to an adversarial sample patch attack.

[0009] Methods for solving problems

[0010] The attack detection device of the present invention includes: an abnormal pattern detection unit that detects whether an abnormal pattern is included in a time series recognition score, wherein the time series recognition score is time series data generated using multiple recognition scores, the multiple recognition scores being calculated using multiple image data obtained by capturing a range within the shooting range at different times within the shooting time range, and representing the result of detecting objects in the multiple image data respectively, and the abnormal pattern being generated when an adversarial sample patch attack is performed on at least one of the multiple image data.

[0011] Invention Effects

[0012] The attack detection apparatus of the present invention includes an anomaly pattern detection unit that detects whether an anomaly pattern generated under conditions where an adversarial sample patching attack is committed is included in a time-series recognition score consisting of a recognition score representing the result of detecting an object. Therefore, according to the present invention, it is possible to detect an adversarial sample patching attack even when such an attack has been performed. Attached Figure Description

[0013] Figure 1 This is a diagram illustrating an example of the system structure of the adversarial sample patch detection system 100 according to Embodiment 1.

[0014] Figure 2 This is a diagram illustrating an example of the functional configuration of the object detection device 110 according to Embodiment 1.

[0015] Figure 3 This is a diagram illustrating an example of the functional structure of the attack detection device 120 according to Embodiment 1.

[0016] Figure 4 This is a diagram showing an example of the hardware structure of the object detection device 110 and the attack detection device 120 in Embodiment 1.

[0017] Figure 5 This is a flowchart illustrating the operation of the adversarial sample patch detection system 100 in Implementation 1.

[0018] Figure 6 This is a diagram showing examples of the hardware structure of the object detection device 110 and the attack detection device 120, which are variations of Embodiment 1. Detailed Implementation

[0019] In the description and accompanying drawings of the embodiments, the same symbols are used to label the same elements and corresponding elements. The descriptions of elements labeled with the same symbols are omitted or simplified as appropriate. The arrows in the figures mainly indicate data flow or processing flow. In addition, the word "section" may be appropriately rewritten as "circuit", "process", "step", "processing" or "line".

[0020] Implementation method 1.

[0021] Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings.

[0022] ***Structure Description***

[0023] Figure 1 An example system structure of the adversarial sample patch detection system 100 according to this embodiment is shown. The adversarial sample patch detection system 100 includes an object detection device 110 and an attack detection device 120. The object detection device 110 and the attack detection device 120 may also be integrally formed.

[0024] The object detection device 110 receives input image data x as input and outputs a recognition score y as the object detection result.

[0025] The attack detection device 120 receives the recognition score y as input and the detection result r of the adversarial sample patch attack. If an anomalous pattern of the recognition score of the adversarial sample patch attack is detected, the output indicates that the adversarial sample patch attack has been detected; otherwise, the output indicates that the adversarial sample patch attack has not been detected. An adversarial sample patch attack is a type of adversarial attack, specifically designed to circumvent object detection methods such as neural networks. As a specific example, an anomalous pattern is a pattern where the recognition score value remains slightly below the object detection threshold for a certain period. That is, an anomalous pattern represents a pattern where the recognition score value shown in the time-series recognition score is less than the object detection threshold and remains above the anomalous detection threshold for a period exceeding the anomalous detection time. In current attacks using adversarial sample patches, the recognition score value can be suppressed below the object detection threshold, but not completely zero. Therefore, this anomalous pattern is effective in detecting this attack. However, the anomalous pattern is not limited to this specific pattern.

[0026] Figure 2 An example of the functional structure of the object detection device 110 is shown. The object detection device 110 includes a data input unit 111, an object detector 112, and a data output unit 113.

[0027] The data input unit 111 receives input image data x, which is the object to be detected, and inputs the received input image data x into the object detector 112.

[0028] Object detector 112 calculates a recognition score y using the input image data x and outputs the calculated recognition score y to data output unit 113. Specifically, object detector 112 is constructed using a neural network. Examples of neural networks include YOLO (You Only Look Once), SSD (Single Shot Multibox Detector), or Faster R-CNN (Region-based Convolutional Neural Networks). Object detector 112 outputs a recognition score, representing the coordinates of the bounding boxes corresponding to each object in the input image and the probability of the object's type and confidence level within each bounding box. Object detector 112 calculates multiple recognition scores using multiple image data. Object detector 112 is also referred to as an object detection unit.

[0029] The data output unit 113 outputs the recognition score y calculated by the object detector 112.

[0030] Figure 3 An example of the functional structure of the attack detection device 120 is shown. The attack detection device 120 includes a data input unit 121, an identification score accumulation unit 122, an abnormal pattern detection unit 123, and a data output unit 124.

[0031] The data input unit 121 receives the recognition score y and inputs the received recognition score y into the recognition score accumulation unit 122.

[0032] The recognition score accumulation unit 122 generates a time-series recognition score Y by appending the input recognition score y to the time-series recognition score Y', and inputs the generated time-series recognition score Y to the abnormal pattern detection unit 123. The time-series recognition score Y is equivalent to the updated time-series recognition score Y'. The time-series recognition score Y' is time-series data generated by accumulating the recognition scores y input before a new recognition score y is input. The recognition score accumulation unit 122 may also choose not to input the time-series recognition score Y to the abnormal pattern detection unit 123 every time a recognition score y is input to the recognition score accumulation unit 122, or the abnormal pattern detection unit 123 may retrieve the latest time-series recognition score Y from the recognition score accumulation unit 122 at regular intervals and use the retrieved time-series recognition score Y. The time-series recognition score is time-series data generated using multiple recognition scores. The multiple recognition scores are calculated using multiple image data obtained by taking pictures of a range within the shooting range at different times within the shooting time range, and represent the results of detecting objects in the multiple image data respectively.

[0033] The anomaly detection unit 123 matches the time-series recognition score Y with a pre-specified anomaly pattern. As a detection result r, if the time-series recognition score Y matches the anomaly pattern, the anomaly detection unit 123 inputs a result indicating the detected pattern to the data output unit 124; if the time-series recognition score Y does not match the anomaly pattern, the anomaly detection unit 123 inputs a result indicating that no adversarial sample patch attack was detected to the data output unit 124. When the anomaly detection unit 123 detects a shift in the recognition score that differs from the normal shift, i.e., an anomaly pattern, it determines that an adversarial sample patch attack has been performed. The anomaly detection unit 123 detects whether the time-series recognition score contains an anomaly pattern generated when at least one of multiple image data has been subjected to an adversarial sample patch attack.

[0034] The data output unit 124 outputs the input detection result r.

[0035] Figure 4 This diagram illustrates an example of the hardware resources of the object detection device 110 and the attack detection device 120 in this embodiment. The object detection device 110 and the attack detection device 120 are each configured as a computer; alternatively, they may be configured as multiple computers.

[0036] Object detection device 110 and attack detection device 120 each have a processor 11. The processor 11 is connected via bus 12 to hardware devices such as ROM 13, RAM 14, communication board 15, display 51 (as a display device), keyboard 52, mouse 53, driver 54, and disk drive 20, and controls these hardware devices. The processor 11 is an integrated circuit (IC) that performs computational processing; specifically, it is a CPU (Central Processing Unit), DSP (Digital Signal Processor), or GPU (Graphics Processing Unit). Object detection device 110 and attack detection device 120 may each have multiple processors. Multiple processors share the functions of processor 11.

[0037] Drive 54 is a device for reading and writing storage media such as FD (Flexible Disk Drive), CD (Compact Disc), or DVD (Digital Versatile Disc).

[0038] ROM13, RAM14, disk drive 20, and drive 54 are examples of storage devices. Storage devices can be independent of the computer.

[0039] Keyboard 52, mouse 53, and communication board 15 are examples of input devices. Display 51 and communication board 15 are examples of output devices.

[0040] The communication board 15 is connected to a communication network such as a LAN (Local Area Network), the Internet, or a telephone line via wired or wireless means. Specifically, the communication board 15 consists of a communication chip or a NIC (Network Interface Card).

[0041] The disk device 20 stores an OS (operating system) 21, a program group 22, and a file group 23.

[0042] Program group 22 contains programs that perform the functions described as components or devices in this embodiment. The programs are read and executed by processor 11. That is, the programs enable the computer to function as components or devices, and further enable the computer to perform the steps or methods of the components or devices.

[0043] Any program described in this specification can be recorded onto a computer-readable, non-volatile recording medium. Specifically, the non-volatile recording medium may be an optical disc or flash memory. Any program described in this specification may be provided as a program product.

[0044] File group 23 contains various data used in the various parts or devices described in this embodiment.

[0045] ***Instructions for Action***

[0046] The operation steps of the object detection device 110 are equivalent to an object detection method. Furthermore, the program that implements the operation of the object detection device 110 is equivalent to an object detection program. The operation steps of the attack detection device 120 are equivalent to an attack detection method. Furthermore, the program that implements the operation of the attack detection device 120 is equivalent to an attack detection program.

[0047] Figure 5 This is a flowchart illustrating an example of the processing of the adversarial sample patch detection system 100 according to this embodiment. The processing of the adversarial sample patch detection system 100 will be explained with reference to this figure.

[0048] (Step S11)

[0049] The data input unit 111 receives input image data x and inputs the received input image data x into the object detector 112.

[0050] (Step S12)

[0051] Object detector 112 uses the input image data x to calculate the recognition score y.

[0052] (Step S13)

[0053] The data output unit 113 outputs the calculated recognition score y.

[0054] (Step S14)

[0055] The data input unit 121 receives the recognition score y and inputs the received recognition score y into the recognition score accumulation unit 122.

[0056] (Step S15)

[0057] The recognition score accumulation unit 122 updates the time series recognition score Y' to the time series recognition score Y by appending the input recognition score y to the time series recognition score Y', and inputs the updated time series recognition score Y to the abnormal pattern detection unit 123.

[0058] (Step S16)

[0059] The abnormal pattern detection unit 123 determines whether the input time series recognition score Y conforms to a pre-specified abnormal pattern.

[0060] If the abnormal pattern matches the time series identification score Y, the attack detection device 120 proceeds to step S17. Otherwise, the attack detection device 120 proceeds to step S18.

[0061] (Step S17)

[0062] The data output unit 124 outputs the result r, which indicates that an adversarial sample patch attack has been detected.

[0063] (Step S18)

[0064] The data output unit 124 outputs a result r indicating that no adversarial sample patch attack was detected.

[0065] ***Explanation of the effects of Implementation Method 1***

[0066] As described above, according to this embodiment, adversarial sample patch attacks can be detected by determining whether the time series identification score conforms to an abnormal pattern.

[0067] ***Other Structures***

[0068] <Variation Example 1>

[0069] Figure 6 Examples of the hardware structures of the object detection device 110 and the attack detection device 120 in this modified example are shown.

[0070] The object detection device 110 and the attack detection device 120 each have a processing circuit 18 to replace the processor 11, the processor 11 and ROM 13, the processor 11 and RAM 14, or the processor 11, ROM 13 and RAM 14.

[0071] The processing circuit 18 is hardware that implements at least a portion of the components of the object detection device 110 and the attack detection device 120.

[0072] The processing circuit 18 can be dedicated hardware, or it can be a processor that executes programs stored in the disk device 20.

[0073] In the case where the processing circuit 18 is dedicated hardware, as a specific example, the processing circuit 18 is a single circuit, a composite circuit, a programmable processor, a parallel programmable processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.

[0074] The object detection device 110 and the attack detection device 120 may each have multiple processing circuits that replace the processing circuit 18. The multiple processing circuits share the functions of the processing circuit 18.

[0075] In the object detection device 110 and the attack detection device 120, some of their functions may be implemented through dedicated hardware, while the remaining functions may be implemented through software or firmware.

[0076] As a specific example, the processing circuit 18 is implemented by hardware, software, firmware, or a combination thereof.

[0077] The processor 11, ROM 13, RAM 14, and processing circuit 18 are collectively referred to as the "processing circuit". That is, the functions of each functional structural element of the object detection device 110 and the attack detection device 120 are realized through the processing circuit.

[0078] ***Other Implementation Methods***

[0079] Implementation method 1 has been described, but multiple parts of this implementation method can also be combined. Alternatively, this implementation method can be implemented in part. Furthermore, this implementation method can be modified as needed and can be implemented in any combination, either as a whole or in part. The parts described as individual components or devices can be installed via firmware, software, hardware, or any combination thereof.

[0080] Furthermore, the above-described embodiments are essentially preferred examples and are not intended to limit the scope of the invention, its applications, and uses. The steps described using flowcharts or similar methods may also be appropriately modified.

[0081] Label Explanation

[0082] 11: Processor; 12: Bus; 13: ROM; 14: RAM; 15: Communication board; 18: Processing circuit; 20: Disk drive; 21: OS; 22: Program group; 23: File group; 51: Display; 52: Keyboard; 53: Mouse; 54: Driver; 100: Adversarial sample patch detection system; 110: Object detection device; 111: Data input unit; 112: Object detector; 113: Data output unit; 120: Attack detection device; 121: Data input unit; 122: Identification score accumulation unit; 123: Abnormal pattern detection unit; 124: Data output unit.

Claims

1. An attack detection device, the attack detection device comprising: The abnormal pattern detection unit detects whether the time series recognition score contains abnormal patterns. The time-series recognition score is time-series data generated using multiple recognition scores. These multiple recognition scores are calculated using multiple image data obtained by capturing images of a range within the shooting range at different times within the shooting time range, and represent the results of detecting objects in the multiple image data. The abnormal pattern is generated when an adversarial sample patch attack is applied to at least one of the multiple image data. The abnormal pattern refers to a pattern in which the value of the recognition score shown in the time series recognition score is less than the object detection threshold and is above the abnormal detection threshold for a period of time that is continuously abnormal for more than the time specified in the abnormal detection score.

2. An adversarial sample patch detection system, the adversarial sample patch detection system having: The attack detection device according to claim 1; and An object detection device having an object detection unit that calculates multiple recognition scores using the multiple image data respectively.

3. An attack detection method, wherein, Computer detection checks whether anomalous patterns are present in time series recognition scores. The time-series recognition score is time-series data generated using multiple recognition scores. These multiple recognition scores are calculated using multiple image data obtained by capturing images of a range within the shooting range at different times within the shooting time range, and represent the results of detecting objects in the multiple image data. The abnormal pattern is generated when an adversarial sample patch attack is applied to at least one of the multiple image data. The abnormal pattern refers to a pattern in which the value of the recognition score shown in the time series recognition score is less than the object detection threshold and is above the abnormal detection threshold for a period of time that is continuously abnormal for more than the time specified in the abnormal detection score.

4. A computer-readable recording medium containing an attack detection program that causes an attack detection device, acting as a computer, to perform anomaly detection processing. In this abnormal pattern detection process, it is determined whether the time series recognition score contains an abnormal pattern. The time-series recognition score is time-series data generated using multiple recognition scores. These multiple recognition scores are calculated using multiple image data obtained by capturing images of a range within the shooting range at different times within the shooting time range, and represent the results of detecting objects in the multiple image data. The abnormal pattern is generated when an adversarial sample patch attack is applied to at least one of the multiple image data. The abnormal pattern refers to a pattern in which the value of the recognition score shown in the time series recognition score is less than the object detection threshold and is above the abnormal detection threshold for a period of time that is continuously abnormal for more than the time specified in the abnormal detection score.

Images