Image processing device, image processing method, and image processing program

Abstract

An image processing device (100) invalidates an attack that escapes object detection in image data (51). A detection unit (120) calculates a first detection result (52) including the score value and an object region indicating an object detected in the image data (51). A working unit (130) extracts, as a first filling candidate (53), an object region which is in the object region included in the first detection result (52), and in which the score value falls within the range of a score threshold (71). A selection unit (140) generates a second filling candidate (54) obtained by removing, from the object region included in the first filling candidate (53), an object region having an area exceeding an area threshold (72). The selection unit (140) divides the object region included in the second filling candidate (54) into a plurality of subsets, merges object regions of the subsets, and generates a final filling candidate (55).

Description

Image processing device, image processing method, and image processing program

[0001] The present disclosure relates to an image processing device, an image processing method, and an image processing program, and more particularly to an image processing device, an image processing method, and an image processing program that achieve high speed in a technique for defeating adversarial patch attacks.

[0002] Non-Patent Document 1 proposes an attack method to evade detection by object detection. Specifically, this attack method is carried out against an object detection task in which the position of each object in an input image is indicated by a bounding box and the type is indicated by a label. In this attack method, perturbations are added to local areas of the input image, or adversarial patches, in which perturbations are printed as images, are placed near objects in the input image. This achieves an attack method to evade detection by object detection. Meanwhile, Patent Document 2 proposes a countermeasure against attacks using adversarial patches. Patent Document 2 discloses a technology that reduces the effect of adversarial patches by filling in bounding boxes with score values ​​within a certain range of sizes, thereby enabling accurate object detection results to be obtained.

[0003] Simen Thys, Wiebe Van Ranst, Toon Goedeme, "Fooling Automated Surveillance Cameras: Adversarial Patches to Attack Person Detection," in The Bright and Dark Sides of Computer Vision: Challenges and Opportunities for Privacy and Security (CV-COPS) (2019). Yoshihiro Koseki, "Nullification Techniques for Adversarial Sample Patch Attacks in Object Detection," JSAI Annual Conference (2023).

[0004] In the countermeasure using filling in non-patent document 2, it is necessary to input the same number of images generated by filling into the object detection model and process them again, which poses a problem of a large increase in the amount of processing compared to normal object detection.

[0005] The present disclosure aims to reduce the amount of processing required in countermeasure processing against adversarial patch attacks by reducing the amount of images generated by filling.

[0006] The image processing device according to the present disclosure is an image processing device that neutralizes attacks that evade object detection in image data, and includes: a detection unit that performs an object detection process to detect an object in the image data and calculates a first detection result including an object region indicating an object detected in the image data and a score value indicating the reliability of the object indicated by the object region; a processing unit that extracts, from the object regions included in the first detection result, object regions whose score values ​​fall within a score threshold range as first fill candidates; and a selection unit that generates second fill candidates by removing object regions having an area exceeding an area threshold from the object regions included in the first fill candidates, divides the object regions included in the second fill candidates into a plurality of subsets, and merges the object regions of each of the plurality of subsets to generate final fill candidates.

[0007] The image processing device according to the present disclosure has the effect of reducing the amount of processing required for countermeasure processing against adversarial patch attacks by reducing the amount of images generated by filling in.

[0008] Fig. 1 is a diagram showing a configuration example of an image processing device according to embodiment 1. Fig. 2 is a diagram showing an example of functional elements and data input / output of an image processing device according to embodiment 1. Fig. 3 is a flow diagram showing a processing example of an image processing device according to embodiment 1. Fig. 4 is a flow diagram showing a processing example of a selection unit according to embodiment 1. Fig. 5 is a diagram showing a configuration example of an image processing device according to a modified example of embodiment 1.

[0009] The present embodiment will be described below with reference to the drawings. In each drawing, identical or corresponding parts are designated by the same reference numerals. In the description of the embodiment, the description of identical or corresponding parts will be omitted or simplified as appropriate. Arrows in the drawings mainly indicate the flow of data or the flow of processing. Furthermore, the sized relationships between components in the following drawings may differ from the actual relationships. Furthermore, in the description of the embodiment, directions or positions such as up, down, left, right, front, rear, front and back may be indicated. These notations are used for convenience of explanation and do not limit the placement, direction or orientation of devices, instruments, parts, etc.

[0010] Embodiment 1. ***Description of Configuration*** Fig. 1 is a diagram showing an example of the configuration of an image processing device 100 according to this embodiment. The image processing device 100 is a computer. The image processing device 100 includes a processor 910, as well as other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware via signal lines and controls this other hardware.

[0011] The image processing device 100 includes, as functional elements, an acquisition unit 110, a detection unit 120, a processing unit 130, a selection unit 140, an integration unit 150, an output unit 160, and a storage unit 170. The storage unit 170 stores a score threshold 71 and an area threshold 72.

[0012] The functions of the acquisition unit 110, the detection unit 120, the processing unit 130, the selection unit 140, the integrating unit 150, and the output unit 160 are realized by software. The storage unit 170 is provided in the memory 921. Note that the storage unit 170 may be provided in the auxiliary storage device 922, or may be provided separately in the memory 921 and the auxiliary storage device 922.

[0013] The processor 910 is a device that executes an image processing program. The image processing program is a program that realizes the functions of the acquisition unit 110, the detection unit 120, the processing unit 130, the selection unit 140, the integration unit 150, and the output unit 160. The processor 910 is an IC that performs arithmetic processing. Specific examples of the processor 910 are a CPU, a DSP, and a GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0014] The memory 921 is a storage device that temporarily stores data. Specific examples of the memory 921 are SRAM and DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory. The auxiliary storage device 922 is a storage device that saves data. A specific example of the auxiliary storage device 922 is an HDD. The auxiliary storage device 922 may also be a portable storage medium such as an SD (registered trademark) memory card, CF, NAND flash, flexible disk, optical disk, compact disk, Blu-ray (registered trademark) disk, or DVD. Note that HDD is an abbreviation for Hard Disk Drive. SD (registered trademark) is an abbreviation for Secure Digital. CF is an abbreviation for CompactFlash (registered trademark). DVD is an abbreviation for Digital Versatile Disk.

[0015] The input interface 930 is a port connected to an input device such as a mouse, keyboard, or touch panel. Specifically, the input interface 930 is a USB terminal. The input interface 930 may also be a port connected to a LAN. USB is an abbreviation for Universal Serial Bus. LAN is an abbreviation for Local Area Network.

[0016] The output interface 940 is a port to which a cable of an output device such as a display is connected. Specifically, the output interface 940 is a USB terminal or an HDMI (registered trademark) terminal. Specifically, the display is an LCD. The output interface 940 is also called a display interface. HDMI (registered trademark) is an abbreviation for High Definition Multimedia Interface. LCD is an abbreviation for Liquid Crystal Display.

[0017] The communication device 950 has a receiver and a transmitter. The communication device 950 is connected to a communication network such as a LAN, the Internet, a telephone line, or Wi-Fi (registered trademark). Specifically, the communication device 950 is a communication chip or NIC. NIC is an abbreviation for Network Interface Card.

[0018] The image processing program is executed in the image processing device 100. The image processing program is read into the processor 910 and executed by the processor 910. The memory 921 stores not only the image processing program but also an OS. OS is an abbreviation for Operating System. The processor 910 executes the image processing program while running the OS. The image processing program and the OS may be stored in an auxiliary storage device 922. The image processing program and the OS stored in the auxiliary storage device 922 are loaded into the memory 921 and executed by the processor 910. Note that part or all of the image processing program may be incorporated into the OS.

[0019] The image processing device 100 may include a plurality of processors that replace the processor 910. These processors share the task of executing the image processing program. Each processor is a device that executes the image processing program in the same way as the processor 910.

[0020] Data, information, signal values ​​and variable values ​​used, processed or output by the image processing program are stored in memory 921, auxiliary storage device 922, or registers or cache memory within processor 910.

[0021] The "unit" of each of the acquisition unit 110, detection unit 120, processing unit 130, selection unit 140, integration unit 150, and output unit 160 may be interpreted as a "circuit," "step," "procedure," "process," or "circuitry." The image processing program causes a computer to execute an acquisition process, a detection process, a processing process, a selection process, an integration process, and an output process. The "processing" of the acquisition process, the detection process, the processing process, the selection process, the integration process, and the output process may be interpreted as a "program," a "program product," a "computer-readable storage medium storing a program," or a "computer-readable recording medium recording a program." The image processing method is a method performed by the image processing device 100 executing the image processing program. The image processing program may be provided by being stored in a computer-readable recording medium. The image processing program may also be provided as a program product.

[0022] *** Functional Overview *** Here, an overview of image processing by the image processing device 100 according to this embodiment will be described. The image processing device 100 according to this embodiment is a device that neutralizes attacks that evade object detection in image data 51. As described above, one type of attack that evades object detection in image data 51 is the adversarial patch attack.

[0023] In countermeasure processing against adversarial patch attacks, the position of an adversarial sample patch in image data 51 is estimated from the score value of the bounding box output by an object detector for the input image data 51. When filling in the position of the adversarial sample patch, the image processing device 100 selects and merges areas of fill candidates. The object detector outputs a bounding box, label, and score value for each object in the image data 51. The bounding box for each object is calculated as coordinates representing the position of the bounding box. The label represents the type of object within the bounding box. The score value is a probability as the confidence level of the bounding box. In other words, the score value represents the reliability of the bounding box.

[0024] The image processing device 100 acquires a bounding box and a score value from the image data 51 using an object detector. The image processing device 100 designates a bounding box as a fill candidate if the score value falls within a certain threshold range. The image processing device 100 then first selects fill candidates based on their area. Specifically, the image processing device 100 removes from the fill candidates any bounding box whose area exceeds a certain threshold.

[0025] Next, the image processing device 100 performs filling on the area obtained by merging the bounding boxes of the fill candidates, which is the fill area. Merging does not necessarily involve merging all bounding boxes into a single area. The image processing device 100 may divide the bounding boxes of the fill candidates into several subsets and merge them to generate multiple fill areas. Filling may involve filling the entire area uniformly with a single color, or may involve filling with a striped or checkered pattern. After filling, the image processing device 100 inputs the filled images back into the object detector and calculates the output for each. The image processing device 100 then performs an integration process on the bounding box group, which combines the output for the image before filling and the output for each image after filling. The image processing device 100 performs an integration process, such as deleting unnecessary bounding boxes, and outputs the result as the final detection result.

[0026] ***Explanation of Operation*** Next, the operation of image processing device 100 according to this embodiment will be described. The operation procedure of image processing device 100 corresponds to an image processing method. Furthermore, a program that realizes image processing, which is the operation of image processing device 100, corresponds to an image processing program.

[0027] Fig. 2 is a diagram showing an example of functional elements and data input / output of the image processing device 100 according to this embodiment. Fig. 3 is a flow chart showing an example of processing of the image processing device according to this embodiment.

[0028] In step S11, the acquisition unit 110 acquires image data 51 via the input interface 930. The acquisition unit 110 inputs the image data 51 to the detection unit 120 and the processing unit .

[0029] In step S12, the detection unit 120 performs an object detection process to detect an object in the image data. The detection unit 120 calculates a first detection result 52 including an object region indicating the object detected in the image data 51 and a score value indicating the certainty that the object indicated by the object region will be used in an attack. The object region indicating the object detected in the image data 51 is, for example, a bounding box. Specifically, the process is as follows.

[0030] The detection unit 120 calculates a bounding box, a label, and a score value as a result of object detection in the image data 51. The detection unit 120 includes an object detector. The detection unit 120 inputs the given image data 51 into the object detector and calculates a bounding box, a label, and a score value. In this case, the object detector is, for example, an object detector constructed by a neural network. Examples of such object detectors include YOLO, SSD, and Faster R-CNN. YOLO is an abbreviation for You Only Look Once. SSD is an abbreviation for Single Shot MultiBox Detector. R-CNN is an abbreviation for Region Based Convolutional Neural Networks.

[0031] In step S13, processing unit 130 extracts, from the object regions included in first detection result 52, object regions whose score values ​​fall within the range of score threshold 71 as first filling candidates 53. Specifically, this is done as follows.

[0032] The processing unit 130 receives a first detection result 52, which is the result of object detection for the image data 51, from the detection unit 120. The processing unit 130 lists, from the first detection result 52, bounding boxes whose score values ​​fall within a certain range, for example, a score threshold 71, as first filling candidates 53. The processing unit 130 inputs the first filling candidates 53 to the selection unit 140.

[0033] In step S14, the selection unit 140 generates second filling candidates 54 by removing object regions having areas exceeding the area threshold 72 from the object regions included in the first filling candidates 53. The selection unit 140 removes bounding boxes having areas exceeding the area threshold 72, which is a certain threshold, from the set of first filling candidates 53, and generates the second filling candidates 54.

[0034] In step S15, the selection unit 140 divides the object regions included in the second filling candidate 54 into a plurality of subsets, and merges the object regions of each of the plurality of subsets to generate the final filling candidate 55. Specifically, this is as follows:

[0035] FIG. 4 is a diagram showing an example of processing by the selection unit 140 according to this embodiment. In step S151, the selection unit 140 divides the set of second fill candidates 54 into one or more subsets. In step S152, the selection unit 140 merges bounding boxes in each subset. The selection unit 140 determines the area obtained by merging the bounding boxes in each subset as the final fill candidate. In step S153, the selection unit 140 outputs the final fill candidate as a final fill candidate 55. The selection unit 140 inputs the final fill candidate 55 to the processing unit 130.

[0036] In step S16, the processing unit 130 generates a filled-in image group 56 by filling in the object region in the final fill-in candidate 55. The processing unit 130 fills in the object region in the final fill-in candidate 55 with a single color. Alternatively, the processing unit 130 may fill in the object region in the final fill-in candidate 55 with a pattern. Specifically, this is as follows.

[0037] The processing unit 130 uniformly fills each area of ​​the final fill-in candidates 55 with a single color to generate a filled-in image group 56. Alternatively, the processing unit 130 may fill each area of ​​the final fill-in candidates 55 with a pattern such as a checkerboard or striped pattern. By filling in each area of ​​the final fill-in candidates 55, the processing unit 130 generates one image for each filled-in area. These images form the filled-in image group 56.

[0038] The processing unit 130 inputs the filled-in image group 56 to the detection unit 120. The processing unit 130 also inputs the first detection result 52, which is the result of object detection using a normal threshold for the original image data 51, to the integrating unit 150. The normal threshold is a threshold for normal object detection, that is, object detection when no filling is performed. As described above, the processing unit 130 generates one image each in which each region of the final fill-in candidate 55 is filled in, and inputs the images to the detection unit 120 as the filled-in image group 56. The processing unit 130 also inputs the first detection result 52 for the original image data to the integrating unit 150.

[0039] In step S17, the detection unit 120 performs an object detection process on each image in the filled-in image group 56. The detection unit 120 calculates a second detection result 57 including an object region and a score value of the object region in each image in the filled-in image group 56. Specifically, the process is as follows:

[0040] The detection unit 120 inputs each image in the filled-in image group 56 to an object detector. The detection unit 120 calculates the object region and score value, which are outputs for each image, with the score value threshold set to a normal value, and sets this as a second detection result 57. The detection unit 120 inputs the second detection result 57 to the integrating unit 150.

[0041] <Integration Process> The integration unit 150 uses the first detection result 52 and the second detection result 57 to output the result of object detection in the image data 51, in which attacks such as adversarial patch attacks have been neutralized, as a final detection result 58. Specifically, this is as follows.

[0042] The integration unit 150 performs integration processing by combining the second detection result 57 received from the detection unit 120 with the first detection result 52. The integration processing consists of two processes, process 1 and process 2. The first detection result 52 is the result of object detection performed on the image data 51. The second detection result 57 is the result of object detection performed on each image in the filled-in image group 56.

[0043] <<Process 1>> In step S18, the integration unit 150 performs Process 1 on the first detection result 52 of the image data 51 before filling and the second detection result 57 of the filled-in image group 56. Process 1 involves NMS, which is also performed in normal object detection. NMS is an abbreviation for Non-Maximum Suppression. In other words, for a group of bounding boxes combining the detection results of the pre-filled image and the filled-in image group, the IOU value with other bounding boxes is calculated, starting with the bounding box with the highest score value as the reference, in descending order. IOU is an abbreviation for Intersection Over Union. IOU is the value obtained by dividing the area of ​​the intersection (A∩B) of two bounding boxes A and B by the area of ​​their union (A∪B), and the greater the overlap between the two bounding boxes, the closer the value is to 1. Any bounding box whose IOU value is equal to or greater than a threshold value relative to the reference bounding box is removed from the output. After comparison with all bounding boxes has been performed, the bounding box with the next highest score among the remaining bounding boxes is set as the reference.

[0044] <<Process 2>> In step S19, the integration unit 150 performs Process 2 on the results of Process 1. After all bounding boxes remaining in the output have been used as the reference, Process 2 is performed on the remaining bounding boxes. In Process 2, the IOA value is calculated with the largest area bounding box as the reference, in order, relative to the other bounding boxes. IOA is the area of ​​the intersection (A∩B) of reference bounding box A and bounding box B, divided by the area of ​​B; the greater the proportion of B included in A, the closer the value is to 1. Bounding boxes with an IOA value equal to or greater than a threshold value relative to the reference bounding box are removed from the output, and after comparison with all bounding boxes has been performed, the bounding box with the next largest area from among the remaining bounding boxes is used as the reference.

[0045] After using all of the bounding boxes remaining in the output as a reference, the integrating unit 150 sends the remaining bounding boxes to the output unit 160 as a final detection result 58, which is the final object detection result. The output unit 160 outputs the content received from the integrating unit 150 as the processing result of the image processing device 100. In step S20, the output unit 160 outputs the result of Process 2 as a final detection result 58 to an output device or the like via the output interface 940.

[0046] ***Other Configurations*** In this embodiment, the functions of the acquisition unit 110, the detection unit 120, the processing unit 130, the selection unit 140, the integrating unit 150, and the output unit 160 are realized by software. As a modified example, the functions of the acquisition unit 110, the detection unit 120, the processing unit 130, the selection unit 140, the integrating unit 150, and the output unit 160 may be realized by hardware. Specifically, the image processing device 100 includes an electronic circuit 909 instead of the processor 910.

[0047] 4 is a diagram showing an example of the configuration of an image processing device 100 according to a modified example of this embodiment. The electronic circuit 909 is a dedicated electronic circuit that realizes the functions of the acquisition unit 110, the detection unit 120, the processing unit 130, the selection unit 140, the integration unit 150, and the output unit 160. Specifically, the electronic circuit 909 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is an abbreviation for Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field-Programmable Gate Array.

[0048] The functions of the acquisition unit 110, detection unit 120, processing unit 130, selection unit 140, integration unit 150, and output unit 160 may be realized by a single electronic circuit, or may be realized by distributing them across multiple electronic circuits.

[0049] As another modification, some of the functions of the acquisition unit 110, the detection unit 120, the processing unit 130, the selection unit 140, the integrating unit 150, and the output unit 160 may be realized by electronic circuits, and the remaining functions may be realized by software. Also, some or all of the functions of the acquisition unit 110, the detection unit 120, the processing unit 130, the selection unit 140, the integrating unit 150, and the output unit 160 may be realized by firmware.

[0050] Each of the processor and the electronic circuit is also called processing circuitry. That is, the functions of the acquisition unit 110, the detection unit 120, the processing unit 130, the selection unit 140, the integration unit 150, and the output unit 160 are realized by the processing circuitry.

[0051] ***Description of Effects of the Present Embodiment*** In the image processing device according to the present embodiment, the detection unit calculates a bounding box and a score value from image data, and determines the result as a first detection result. The processing unit determines a bounding box from the first detection result whose score value falls within a certain range as a first fill candidate. The selection unit then selects from the first fill candidates based on area, and determines the second fill candidate. The selection unit divides the second fill candidates into subsets and merges them to generate final fill candidates. The processing unit fills each of the final fill candidates, and passes them to the detection unit as a filled-in image group. The detection unit again performs object detection on each image in the filled-in image group, and determines the second detection result. The integration unit performs processes 1 and 2 on the first detection result of the image data before filling and the second detection result of the filled-in image group. The merging unit removes redundant bounding boxes through processes 1 and 2, and outputs the remaining bounding boxes as correct bounding boxes in which the attack has been nullified. In this way, according to the image processing device of this embodiment, by merging the bounding boxes to be filled, the amount of image generated by filling can be reduced. Therefore, the amount of processing required for countermeasures against adversarial patch attacks can be reduced. However, since simple merging would result in an excessively large fill area, the image processing device of this embodiment selects merge targets using an area-based threshold. This allows the amount of processing to be reduced without reducing the accuracy of countermeasures against adversarial patch attacks.

[0052] In the first embodiment described above, each unit of the image processing device has been described as an independent functional block. However, the configuration of the image processing device does not have to be the same as that of the above-described embodiment. The functional blocks of the image processing device may have any configuration as long as they can realize the functions described in the above-described embodiment. Furthermore, the image processing device may not be a single device, but may be a system composed of multiple devices. Furthermore, multiple parts of the first embodiment may be combined to implement the first embodiment. Alternatively, only one part of the first embodiment may be implemented. In addition, the first embodiment may be combined in any way, either as a whole or in part. In other words, in the first embodiment, the various embodiments may be freely combined, or any component of each embodiment may be modified, or any component of each embodiment may be omitted.

[0053] The above-described embodiments are essentially preferred examples and are not intended to limit the scope of the present disclosure, the scope of application of the present disclosure, or the scope of use of the present disclosure. The above-described embodiments can be modified in various ways as needed. For example, the procedures described using flow charts or sequence diagrams may be modified as appropriate.

[0054] Various aspects of the present disclosure are summarized below as appendices.

[0055] (Supplementary Note 1) An image processing device that neutralizes attacks that evade object detection in image data, comprising: a detection unit that performs an object detection process to detect an object in the image data and calculates a first detection result including an object region that indicates an object detected in the image data and a score value that indicates the reliability of the object indicated by the object region; a processing unit that extracts, from the object regions included in the first detection result, object regions whose score values ​​are within a score threshold range, as first fill-in candidates; and a selection unit that generates second fill-in candidates by removing object regions that have an area that exceeds an area threshold from the object regions included in the first fill-in candidates, divides the object regions included in the second fill-in candidates into a plurality of subsets, and generates final fill-in candidates by merging the object regions of each of the plurality of subsets. (Supplementary Note 2) The image processing device according to Supplementary Note 1, wherein the processing unit generates a group of filled-in images by filling in the object region in the final fill-in candidates, the detection unit performs the object detection process on each image in the group of filled-in images and calculates a second detection result including the object region in each image in the group of filled-in images and the score value of the object region, and the image processing device includes: an integration unit that uses the first detection result and the second detection result to output a result of object detection in the image data in which the attack has been neutralized as a final detection result. (Supplementary Note 3) The image processing device according to Supplementary Note 2, wherein the processing unit fills in the object region in the final fill-in candidates with a single color. (Supplementary Note 4) The image processing device according to Supplementary Note 2, wherein the processing unit fills in the object region in the final fill-in candidates with a pattern. (Supplementary Note 5) The image processing device according to any one of Supplements 1 to 4, wherein the object region is a bounding box indicating an object detected in the image data.(Supplementary Note 6) An image processing method used in an image processing device that neutralizes attacks that evade object detection in image data, the image processing method comprising: a computer performs object detection processing to detect an object in the image data, and calculates a first detection result including an object region that indicates an object detected in the image data and a score value that indicates the reliability of the object indicated by the object region; the computer extracts, from the object regions included in the first detection result, object regions whose score values ​​are within a score threshold range, as first fill-in candidates; the computer generates second fill-in candidates by removing object regions that have an area exceeding an area threshold from the object regions included in the first fill-in candidates, divides the object regions included in the second fill-in candidates into a plurality of subsets, and merges the object regions of each of the plurality of subsets to generate a final fill-in candidate. (Supplementary Note 7) An image processing program used in an image processing device that neutralizes attacks that evade object detection in image data, the image processing program causing the image processing device, which is a computer, to execute the following steps: a detection process that performs an object detection process that detects an object in the image data and calculates a first detection result including an object region that indicates an object detected in the image data and a score value that indicates the reliability of the object indicated by the object region; a processing process that extracts, from the object regions included in the first detection result, object regions whose score values ​​are within a score threshold range, as first fill-in candidates; and a selection process that generates second fill-in candidates by removing object regions that have an area that exceeds an area threshold from the object regions included in the first fill-in candidates, divides the object regions included in the second fill-in candidates into a plurality of subsets, and merges the object regions of each of the plurality of subsets to generate final fill-in candidates.

[0056] 51 Image data, 52 First detection result, 53 First fill candidate, 54 Second fill candidate, 55 Final fill candidate, 56 Filled image group, 57 Second detection result, 58 Final detection result, 71 Score threshold, 72 Area threshold, 100 Image processing device, 110 Acquisition unit, 120 Detection unit, 130 Processing unit, 140 Selection unit, 150 Integration unit, 160 Output unit, 170 Storage unit, 909 Electronic circuit, 910 Processor, 921 Memory, 922 Auxiliary storage device, 930 Input interface, 940 Output interface, 950 Communication device.

Claims

1. An image processing device that neutralizes attacks that evade object detection in image data, comprising: a detection unit that performs object detection processing to detect objects in the image data and calculates a first detection result including an object region that indicates an object detected in the image data and a score value that indicates the reliability of the object indicated by the object region; a processing unit that extracts, from the object regions included in the first detection result, object regions whose score values ​​fall within a score threshold range as first fill candidates; and a selection unit that generates second fill candidates by removing object regions that have an area that exceeds an area threshold from the object regions included in the first fill candidates, divides the object regions included in the second fill candidates into a plurality of subsets, and generates final fill candidates by merging the object regions of each of the plurality of subsets.

2. The image processing device according to claim 1, further comprising: an integration unit that uses the first detection result and the second detection result to output a result of object detection in the image data in which the attack has been neutralized as a final detection result; wherein the processing unit generates a group of filled-in images in which the object areas in the final filled-in candidates have been filled in; the detection unit performs the object detection process on each image in the group of filled-in images and calculates a second detection result including the object area in each image in the group of filled-in images and the score value of the object area; and wherein the image processing device further comprises: an integration unit that uses the first detection result and the second detection result to output a result of object detection in the image data in which the attack has been neutralized as a final detection result.

3. The image processing device according to claim 2, wherein the processing unit fills the object area in the final fill-in candidate with a single color.

4. The image processing device according to claim 2, wherein the processing unit fills the object area in the final fill-in candidate with a pattern.

5. An image processing device according to any one of claims 1 to 4, wherein the object region is a bounding box representing an object detected in the image data.

6. An image processing method used in an image processing device that neutralizes attacks that evade object detection in image data, wherein a computer performs object detection processing to detect objects in the image data and calculates a first detection result including an object region that indicates an object detected in the image data and a score value that indicates the reliability of the object indicated by the object region; the computer extracts, from the object regions included in the first detection result, object regions whose score values ​​fall within a score threshold range as first fill candidates; the computer generates second fill candidates by removing object regions that have an area exceeding an area threshold from the object regions included in the first fill candidates; divides the object regions included in the second fill candidates into a plurality of subsets; and merges the object regions of each of the plurality of subsets to generate final fill candidates.

7. An image processing program used in an image processing device that neutralizes attacks that evade object detection in image data, the image processing program causing the image processing device, which is a computer, to execute the following steps: a detection process that performs object detection processing to detect objects in the image data and calculates a first detection result including an object area that indicates an object detected in the image data and a score value that indicates the reliability of the object indicated by the object area; a processing process that extracts, from the object areas included in the first detection result, object areas whose score values ​​fall within a score threshold range as first fill-in candidates; and a selection process that generates second fill-in candidates by removing object areas that have an area exceeding an area threshold from the object areas included in the first fill-in candidates, divides the object areas included in the second fill-in candidates into a plurality of subsets, and merges the object areas of each of the plurality of subsets to generate final fill-in candidates.

Images